[mnet-devel] Further EGTP Questions

[Zooko O'Whielacronx]

> Let's say we have 10 million user nodes and we're running the trackers.

I think you mean "relay servers" by "trackers" here.

> Let's say we have 10 million user nodes and we're running the trackers.
>
> Further, let's say that users are "bursty" in the sense that most of 
> the time they are sending one or two transmissions per hour to other 
> nodes, but for a couple of hours a day they are sending a transmission 
> every 30 seconds.  That's 22 * 2 + 2 * 60 * 2 = 44 + 240 = 288 
> transmissions / day / node.
>    A transmission might be any size from a few bytes up to a few hundred 
> Kb, with the shorter ones being a lot more frequent.
>    Any given node is likely to have a few other nodes that it regularly 
> communicates with, which will only change slowly.

Given your description of the probable behavior, I think the limiting factor is 
the case when lots of nodes are sending msgs every 30 seconds.  Let's guess that 
the heaviest load is when 5 million users are *all* sending a msg every 30 s,
and the mean msg size is 2 KB. 

That's probably a more difficult network behavior than would happen in practice, 
so this is kind of a worst-case scenario.

Furthermore, let's say all 5 million of these users are behind NAT or firewalls 
so that they can't accept incoming TCP connections!

Then they are generating an aggregate traffic of 10 GB/s.

If you want to proxy all that traffic through your servers I think it will be
somewhat expensive for you.

> 1) Are there any scalability issues involved in running a tracker?  How 
> many user nodes can a single tracker support, given 288 transmissions / 
> node / day?

I guess the issue isn't the server you run, but instead the bandwidth your 
service requires, whether you operate a single server or hundreds.

I don't have much experience with such high-traffic networking in practice.  
Other people on this list (notably Jim McCoy) do.

> 2a) Looking down the road a ways, how difficult would it be to modify 
> EGTP to keep a connection open between two nodes so that messages could 
> be sent back and forth repeatedly?

It currently does this.

> 2b) Is there any possibility of retooling EGTP for streaming data 
> between nodes, or would we be better advised to consider a different 
> library?

I think EGTP as it currently exists is a good substrate for this, but not for 
latency/jitter-sensitive streaming such as VoIP.  Perhaps future variations of 
EGTP could be more suitable for VoIP.

Anyway, if you don't worry about latency and jitter then it is easy to call 
Node.send() a hundred times, each time sending another 1/100th of the file.

> Let's say that, like a lot of people, I'm sharing my internet 
> connection with one of those little 4 port router + firewall + wireless 
> boxes.  They're pretty common now, and a lot of our users are going to 
> have one.
> 
> 3a) Does using one mean that I can't run a relay node?

Pretty much.  It does mean that you can't accept incoming TCP connections.  
I recently realized that such servers might actually be useful as relay servers, 
but let's only get into that if we need to.  :-)

> 3b) ...unless I open the appropriate ports in the firewall?

Yes.

http://mnet.sourceforge.net/faq.php#behind_firewall

--Z

-------
http://zooko.com/
Please excuse if the above message is terse; my arm is sore from too much typing.



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
mnet-devel mailing list
mnet-devel at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mnet-devel