[mnet-devel] how to convert to twisted (was: What's left to be done for 0.7)

[Zooko O'Whielacronx]

Nowadays I find the idea of a whole-scale conversion to the entire twisted 
stack very appealing.

This wouldn't take as long as you think because:

(a) a great deal of code can just be abandoned (at least for now) and excluded 
from the conversion.  I'll do some quantitative analysis of that in a minute 
and append it to the end of this letter.

(b) there are no significant semantic gaps between our current stuff 
(MojoTransaction, DoQ, EGTP, TCPCommsHandler) and the twisted equivalents (PB, 
reactor, SSL or cleartext, twisted.transport.TCP).

Actually there is an exception to (b), which is that twisted doesn't have a 
comms facility for message-based (not stream-based) encryption.  v0.6 uses 
such a facility for relay service.  In v0.7 I believe that UPnP and/or ent-
routes-around-firewalls will replace relay service, so we won't need that 
facility in v0.7.0-STABLE.

Now I expect to want such a facility again soon, perhaps in v0.7.1 or v0.8, in 
which case we could port EGTP as a twisted transport layer or find some other 
such solution.


Major reason to delay this large-scale conversion to later:  the current stuff 
works, and the conversion takes time and is risky.

Major reason to do it sooner rather than later:  converting to the *whole* 
Twisted stack, from PB down to reactor and everything in between, means 
switching to banana/spread instead of mencode.  This breaks backwards 
compatibility with existing nodes.  I don't want to go to the effort of 
putting out a dual-mode "bridge" release.  Right now there is next to no 
content on the network and very few users.  Breaking backwards compatibility 
ASAP hurts less than breaking it later.


I know this is a contentious issue for a lot of folks.


Here's my quick estimate of what code we would have to port for a fully-
Twistified version of Mnet.  Here's the top-level directories with number of 
unique lines of code:

HACK pion:~/playground/mnet/mnet_newnet$ for D in `find . -type d -mindepth 1 -maxdepth 1`; do echo ${D};  if [ "X`find ${D} -name '*.py' -or -name '*.c' -or -name '*.h' -or -name '*.cpp'`" != "X" ] ; then cat `find ${D} -name '*.py' -or -name '*.c' -or -name '*.h' -or -name '*.cpp' ` | sort -u | wc -l; fi ; done

.
  16673
./CVS
./doc
./w32
    150
./egtp
   6121
./mnetlib
   8815
./artwork
./evilcrypto
   1502

Now we can exclude egtp/ and evilcrypto/ right off the bat.  w32/ is a few 
launching/OS-integration things that may or may not be working right now and 
may or may not need to change for Twistification.

So now I'll look at mnetlib/.

.  
   8815
./CVS
./test
    542
./relay
    109
./webui
    110
./wxgui
   2954
./scripts
    531
./tracker
./filesystem
   3053

Relay can go.  I don't know about webui or wxgui.  I would be surprised if 
scripts had to be changed for Twistification.

There are 11 files in mnetlib/*.py.  5 of them import DoQ.  There are 21 lines 
containing the string "doq" (case-insensitive and excluding the import lines).  
Those files are: async_http_get.py, bootpage.py, datablock.py, peerset.py, 
underboss.py.

test/ *would* have to be changed.  There are 3 files which import DoQ, and a 
total of 18 lines containing the string "doq".

Now to the filesystem/ directory.

There 14 files in filesystem/.  Six of them import DoQ: localblockstore.py, 
pusher.py, storedeep.py, storedict.py, storeflat.py, storehash.py.  There are 
12 lines that contain "doq".


Regards,

Zooko



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
mnet-devel mailing list
mnet-devel at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mnet-devel