Comments on: User space traffic shaping with Ruby http://www.bugfree.dk/blog/2008/04/12/user-space-traffic-shaping-with-ruby/ Not anti-anything, just pro-quality Thu, 16 Dec 2010 07:18:58 +0000 hourly 1 http://wordpress.org/?v=3.3 By: Bugfree.dk » Blog Archive » Computer activation with Asp.Net MVC http://www.bugfree.dk/blog/2008/04/12/user-space-traffic-shaping-with-ruby/comment-page-1/#comment-21982 Bugfree.dk » Blog Archive » Computer activation with Asp.Net MVC Tue, 22 Dec 2009 10:21:54 +0000 http://www.bugfree.dk/blog/2008/04/12/user-space-traffic-shaping-with-ruby/#comment-21982 [...] posts in this series include: Kernel space traffic shaping with Linux and User space traffic shaping with Ruby that touch exclusively on Linux [...] [...] posts in this series include: Kernel space traffic shaping with Linux and User space traffic shaping with Ruby that touch exclusively on Linux [...]

]]> By: Bugfree.dk » Blog Archive » Kernel space traffic shaping with Linux http://www.bugfree.dk/blog/2008/04/12/user-space-traffic-shaping-with-ruby/comment-page-1/#comment-16404 Bugfree.dk » Blog Archive » Kernel space traffic shaping with Linux Thu, 16 Jul 2009 21:55:23 +0000 http://www.bugfree.dk/blog/2008/04/12/user-space-traffic-shaping-with-ruby/#comment-16404 [...] In conclusion, a great deal of time went into experimenting with the various qdiscs, even in combination with ipp2p. But eventually we decided that none of the qdiscs were up to solving our network congestion problem. The research effort wasn’t all in vain, though, because we found that WRR provides us with a cost effective way of counting incoming and outgoing bytes on a computer by computer basis. Thus, in a way all this helped crystallize the idea of building a payload agnostic, user space traffic shaper in Ruby. [...] [...] In conclusion, a great deal of time went into experimenting with the various qdiscs, even in combination with ipp2p. But eventually we decided that none of the qdiscs were up to solving our network congestion problem. The research effort wasn’t all in vain, though, because we found that WRR provides us with a cost effective way of counting incoming and outgoing bytes on a computer by computer basis. Thus, in a way all this helped crystallize the idea of building a payload agnostic, user space traffic shaper in Ruby. [...]

]]> By: Murariu Serban http://www.bugfree.dk/blog/2008/04/12/user-space-traffic-shaping-with-ruby/comment-page-1/#comment-14981 Murariu Serban Thu, 26 Mar 2009 09:33:35 +0000 http://www.bugfree.dk/blog/2008/04/12/user-space-traffic-shaping-with-ruby/#comment-14981 yes, just remembered (maybe you can "merge the 2 posts") Ad block plus for Firefox blocks ads (self explanatory) the more people use it in your network, the less bandwidth they will "waste" and the faster their pages would load and the less cluttered their "world" (wide web) will become. Introduce it to your users, (don't enforce it) and most of them will instantly see the benefit and adopt it without further ado. best of luck ! yes, just remembered (maybe you can “merge the 2 posts”)

Ad block plus for Firefox blocks ads (self explanatory) the more people use it in your network, the less bandwidth they will “waste” and the faster their pages would load and the less cluttered their “world” (wide web) will become.

Introduce it to your users, (don’t enforce it) and most of them will instantly see the benefit and adopt it without further ado.

best of luck !

]]> By: Murariu Serban http://www.bugfree.dk/blog/2008/04/12/user-space-traffic-shaping-with-ruby/comment-page-1/#comment-14979 Murariu Serban Thu, 26 Mar 2009 09:23:30 +0000 http://www.bugfree.dk/blog/2008/04/12/user-space-traffic-shaping-with-ruby/#comment-14979 Hey, I really don't quite get this: your goal should be to make the most out of the bandwidth that you have available but all i see here is just a fancy and overengeneered way of limiting traffic per computer per time. Yes it's a nice thing to pull off, bet you learned loads of stuff but in the end is this really it; is this really THE solution? Hardly think so. For instance you should allow someone to continue downloading past the 10Gb limit if you have 40% bandwidth use, right? Moreover: if I'm the only person online then i should get 100% of the bandwidth. I used to run a 400 user network with faaaaaaaar less bandwidth than you guys now have and one of the things that i noticed is that bandwidth usage decreased a hell of a lot after setting up a Dc++ hub ;). Everyone was sharing the movies they downloaded and this meant a lot. Apart from that they would copy the movies at lan speeds and of course that this is preferable to downloading from p2p. I mean it's still p2p but it's local. Of course people still downloaded using eMule/torrentz but the amount of downloaded data was far less and once a movie "entered" the network it was "reused" Also I implemented a transparent squid proxy server that had a dedicated 4.3 gig scsi hard drive for caching. By increasing the default 4k limit for caching objects (ie. allowing it to cache bigger objects) i managed to "save" between 700mb and 1 gig of bandwidth per day (24 hour period), not to mention the "instant" loading speeds. Furthermore, it was configured in such a way that it would block *banner* *ads* etc etc etc and the people loved to read their yahoo mail without annoying ads. You have to realize that i'm talking about 2000~2002 when things like ad block plus were not in existence. The traffic shaping on the routers was done using htb, or more accurately htb-tools. http://htb-tools.skydevel.ro/e107_plugins/content/content.php?content.9 Ip allocation was still dhcp but the ip's were fixed to the mac addresses. We also had a network access system (to be able to restrict access to those who did not pay) etc etc. those were fun days. hope this helps i might have left some things out. if i remember anything i'll post again regards Hey,
I really don’t quite get this: your goal should be to make the most out of the bandwidth that you have available but all i see here is just a fancy and overengeneered way of limiting traffic per computer per time. Yes it’s a nice thing to pull off, bet you learned loads of stuff but in the end is this really it; is this really THE solution? Hardly think so. For instance you should allow someone to continue downloading past the 10Gb limit if you have 40% bandwidth use, right? Moreover: if I’m the only person online then i should get 100% of the bandwidth.
I used to run a 400 user network with faaaaaaaar less bandwidth than you guys now have and one of the things that i noticed is that bandwidth usage decreased a hell of a lot after setting up a Dc++ hub ;). Everyone was sharing the movies they downloaded and this meant a lot. Apart from that they would copy the movies at lan speeds and of course that this is preferable to downloading from p2p. I mean it’s still p2p but it’s local. Of course people still downloaded using eMule/torrentz but the amount of downloaded data was far less and once a movie “entered” the network it was “reused”

Also I implemented a transparent squid proxy server that had a dedicated 4.3 gig scsi hard drive for caching. By increasing the default 4k limit for caching objects (ie. allowing it to cache bigger objects) i managed to “save” between 700mb and 1 gig of bandwidth per day (24 hour period), not to mention the “instant” loading speeds. Furthermore, it was configured in such a way that it would block *banner* *ads* etc etc etc and the people loved to read their yahoo mail without annoying ads. You have to realize that i’m talking about 2000~2002 when things like ad block plus were not in existence.

The traffic shaping on the routers was done using htb, or more accurately htb-tools. http://htb-tools.skydevel.ro/e107_plugins/content/content.php?content.9

Ip allocation was still dhcp but the ip’s were fixed to the mac addresses. We also had a network access system (to be able to restrict access to those who did not pay) etc etc.

those were fun days.

hope this helps

i might have left some things out. if i remember anything i’ll post again

regards

]]> By: Mads Lindstrøm http://www.bugfree.dk/blog/2008/04/12/user-space-traffic-shaping-with-ruby/comment-page-1/#comment-5475 Mads Lindstrøm Sun, 13 Apr 2008 13:15:14 +0000 http://www.bugfree.dk/blog/2008/04/12/user-space-traffic-shaping-with-ruby/#comment-5475 As a former resident, in one of the 330 mentioned apartments, I can testify to the fact that the network ran a lot better after the custom traffic shaper was implemented. A shared network can purchase more and more outbound bandwidth, but people will just start (mis)-using it for huge amounts of p2p traffic. It seems like no amount of bandwidth is enough. I think that a small (below 20) shared Internet connections can work by encouraging users to behave properly, but not a 330 person shared connection. With this many people, there will always be somebody that just cannot see the big picture and follow some simple rules. Here a solution, like the one describe, is the only thing that will work. But I am also a bit curious about performance of the Ruby traffic shaper. How much CPU/memory/harddisk IO do the system use? As a former resident, in one of the 330 mentioned apartments, I can testify to the fact that the network ran a lot better after the custom traffic shaper was implemented. A shared network can purchase more and more outbound bandwidth, but people will just start (mis)-using it for huge amounts of p2p traffic. It seems like no amount of bandwidth is enough.

I think that a small (below 20) shared Internet connections can work by encouraging users to behave properly, but not a 330 person shared connection. With this many people, there will always be somebody that just cannot see the big picture and follow some simple rules. Here a solution, like the one describe, is the only thing that will work.

But I am also a bit curious about performance of the Ruby traffic shaper. How much CPU/memory/harddisk IO do the system use?

]]>