· 10 years ago · Sep 07, 2016, 08:06 PM
1
2
32016-08-18
4Using Freenet over Tor
5
6This post outlines a method of using Freenet over Tor based on posts I wrote on my Freenet hosted blog and subsequent discussions about it. If you read my Freenet hosted blog there's little new here, I'm just making it available on my non-freenet blog.
7
8One issue I've had with Freenet is that it exposes your IP address to peers. Recent law enforcement efforts to monitor Freenet have shown that they have been able to obtain search warrants based on logging requests for blocks of known data and associating them with IP addresses. If law enforcement can do this, so can random bad people.
9
10You can avoid exposing your IP address to random strangers on opennet by using darknet but even then you have to trust your friends aren't monitoring your requests. If it was possible to run Freenet over Tor hidden services then only the hidden service address would be exposed using this logging method. A problem is that Freenet uses UDP which Tor does not support.
11
12A recent post on the Freenet development mailing list pointed out that onioncat provides a virtual network over Tor and tunnels UDP. Using the steps they provided, and some tweaks, it's possible to set up a darknet node that doesn't expose its IP address. It uses the onioncat generated IPv6 address for communicating with peers - and this address is backed by a Tor hidden service.
13
14The steps below outline how to set this up. Note that this is quite experimental and requires care to not expose your IP address. There are some Freenet issues that make things difficult so you should be aware that you do this at your risk and understand it may still expose your identity if things go wrong.
15
16I'm assuming a Debian/Ubuntu like system for the steps.
17Install Tor
18
19Install Tor:
20
21$ sudo apt-get install tor
22
23Edit the /etc/tor/torrc file to enable a Hidden Service with an entry like:
24
25HiddenServiceDir /var/lib/tor/freenet/
26HiddenServicePort 8060 127.0.0.1:8060
27
28Restart Tor and find your hidden service hostname:
29
30$ sudo systemctl restart tor
31$ sudo cat /var/lib/tor/freenet/hostname
32
33Install onioncat
34
35Install onioncat:
36
37$ sudo apt-get install onioncat
38
39Edit /etc/default/onioncat and change the lines matching the following:
40
41ENABLED=yes
42DAEMON_OPTS="-d 0 hiddenservicename.onion -U"
43
44Restart onioncat:
45
46$ sudo systemctl stop onioncat
47$ sudo systemctl start onioncat
48
49Find your onioncat IP address with:
50
51$ ocat -i hiddenservicename.onion
52
53Install Freenet
54
55Install Freenet in the usual way and go through the browser based setup wizard. Choose "Details settings: (custom)" for the security option. On the subsequent pages of the wizard:
56
57 Disable the UPnP plugin.
58 Choose "Only connect to your friends"
59 Choose "High" for "Protection against a stranger attacking you over the internet"
60 Click the "I trust at least one person already using Freenet" checkbox.
61 For "Protection of your downloads..." pick any option you want.
62 Pick a node name that your darknet friends will see.
63 Pick a datastore size that you want.
64 Choose the bandwidth limit.
65
66The node will now be started but have no connections. There will be warnings about this.
67Configure Freenet over Tor
68
69The following settings need to be changed in "Configuration/Core Settings" - make sure you have clicked "Switch to advanced mode".
70
71 Change "IP address override" to your onioncat IP address retrieved in the previous section.
72 Apply the changes.
73
74Shut down Freenet and edit the wrapper.conf file in the Freenet installation directory. Change the line that contains java.net.preferIPv4Stack=true to java.net.preferIPv4Stack=false. In my wrapper.conf this is:
75
76wrapper.java.additional.3=-Djava.net.preferIPv4Stack=false
77
78Edit freenet.ini file in the Freenet installation directory. Change or add the following (replace "onioncat IP address" with the IP address obtained installing onioncat):
79
80node.opennet.bindTo=onioncat IP address
81node.bindTo=onioncat IP address
82node.load.subMaxPingTime=2500
83node.load.maxPingTime=5k
84
85Save the file and restart Freenet. There might be a warning about "Unknown external address". Ignore this as you've explictly set one. I provide a patch later in this post if you want to get rid of the warning.
86Add a friend
87
88Now is the time to add a Darknet friend who is also using Tor/Onioncat. Go to "Friends/Add a friend". Choose your trust and ability to see other friends settings and enter a description of the friend. Paste their noderef in the "Enter node reference directly" box.
89
90Give your noderef to your friend and have them add it. Once both connections have been added you should see "Connected" in the Friends list for that connection. The IP address should show the onioncat IPv6 address, beginning with "fd".
91Optional Freenet patch
92
93When running a Tor based node Freenet thinks the onioncat IP address is a local address. Some places in the Freenet code base check for this and reject it as a valid global routable address. In the FProxy user interface a large warning appears on each page that it couldn't find the external IP address of the node. The other issue is that local addresses aren't counted for bandwidth statistic reporting. The bandwidth box on the statistics page is empty as a result.
94
95I use a patch, onioncat.txt, that provides a workaround for these two issues. The patch is optional as the node works without it but it's a useful improvement if you plan to run a Tor based node long term. You should check the patch before applying it blindly and assure that it's not doing anything nefarious.
96Hybrid nodes
97
98If you run a Tor based darknet node then at least one hybrid node must be in the darknet to bridge to the non-tor nodes. These hybrid nodes will have a public clearnet IP address exposed. I outline how to set up a hybrid node later below. For those that trust me, if you send a darknet tor noderef to me at the freemail address on the bottom of this page, or via normal email, I'll connect and send you a noderef of a hybrid node setup in this manner.
99
100Install Tor and Onioncat as described previously. Install Freenet in the usual way and go through the browser based setup wizard. Choose "Details settings: (custom)" for the security option. On the subsequent pages of the wizard:
101
102 Enable or Disable the UPnP plugin as necessary depending on what you need for your clearnet connection to work.
103 Choose "Connect to strangers"
104 Choose "Low" or "Normal" security as desired.
105 For "Protection of your downloads..." pick any option you want.
106 Pick a datastore size that you want.
107 Choose the bandwidth limit.
108
109The node will start and connect to opennet.
110
111Shut down Freenet and edit the wrapper.conf file in the Freenet installation directory. Change the line that contains java.net.preferIPv4Stack=true to java.net.preferIPv4Stack=false. In my wrapper.conf this is:
112
113wrapper.java.additional.3=-Djava.net.preferIPv4Stack=false
114
115Edit freenet.ini file in the Freenet installation directory. Change or add the following:
116
117node.load.subMaxPingTime=2500
118node.load.maxPingTime=5k
119
120Save the file and restart Freenet. If you base64 decode the "physical.udp" section of the noderef for the node you should see that it now contains the onioncat IP address as well as the public clearnet IP address.
121
122Adding friends to this node will give those friends access to the wider Freenet datastore when they reciprocate.
123
124Don't forget to check your noderefs to ensure that the ARK and the public IP address contain data you are willing to reveal. Check both the darknet noderef and the opennet noderef. You can decode the base64 of the "physical.udp" line with the GNU base64 command:
125
126$ echo "physical.udp base64 here" |base64 -d
127
128Final steps and caveats
129
130Try visiting a Freenet index site and see if it loads. If it does then the Freenet over Tor setup is working. It will be slower than normal Freenet usage due to Tor latency. If you connect to more darknet nodes it will get faster.
131
132When adding a friends noderef you can check what IP addresses it will connect to by looking at the "physical.udp" line. This is a base64 encoded list of IP addresses. You might want to check this to ensure that there are no clearnet addresses in there. If there is a clearnet address then it could deanonymize your node when it tries to connect to that in preference to the onioncat address.
133
134The "ark.pubURI" portion of the noderef is an SSK that points to updated IP address information. A node can subscribe to the USK version of this and learn about IP address changes. Your friends node could change their IP address to a clearnet address resulting in you connecting to that.
135
136To avoid the above two issues it's worthwhile running Freenet in a VM or container that does not have clearnet network access and only has access to the onioncat network setup. Alternatively you can use iptables to only allow onioncat traffic for the Freenet process or user running it.
137
138The IP addresses exposed in the noderef include all local link addresses and their scopes. This is Freenet bug 6879. This may leak information you don't want leaked. It pays to check the "physical.udp" and "ark.pubURI" to see what you are exposing. Remember that any IP addresses exposed over the ARK is discoverable by looking at previous editions of the USK.
139
140The traffic footprint of Freenet may make it easier to track down your IP address from your Tor ID. The volume of data and the nature of the traffic may make certain types of Tor de-anonymization techniques more effective.
141
142Ideally it would be possible to have an opennet of Tor nodes so the exchange of darknet noderefs wouldn't be needed. I haven't been able to get this working yet but I'll continue to investigate it.
143
144I've been running a Tor darknet node for the past week to test how well it works. With three darknet connections it runs well enough for browsing freesites. Sone and the Web of Trust took quite a while to bootstrap due to the lower speed but once it was running it works well. FMS and Flip are also usable. I'd expect performance to be even better with more connections.
145Tags: freenet
146
1472016-08-12
148Bundling Inferno Applications
149
150Inferno can run as a standalone OS or hosted in an existing OS. In the latter case an emu executable runs and executes as a virtual operating system. An application written in Limbo runs inside this virtual OS. When distributing such an application you can install Inferno on the target machine with the application compiled code inside the filesystem of Inferno. This will include a bunch of stuff that the application may not need since it includes all the utilities for the operating system. To distribute the minimal amount of dependencies to run the Limbo application you can create a cut down Inferno distribution that only includes the application dependencies and run that. Another option is to bundle the root filesystem into the executable leaving only that executable to be distributed.
151Application specific Inferno
152
153To create an application specific Inferno install we need to find out what the dependencies are for the application. Most of the following comes from powerman's post on the topic in Russian.
154
155When running emu it runs a program /dis/emuinit.dis to initialize the system and start the shell or other program. An application specific Inferno distribution would require:
156
157 emu
158 /dis/emuinit.dis
159 /dis/lib/emuinit.dis
160 Your application program files inside the Inferno directory structure somewhere.
161
162Powerman's post goes into detail on how to do this with the following "Hello World" Limbo application:
163
164implement HelloWorld;
165include "sys.m";
166include "draw.m";
167
168HelloWorld: module
169{
170 init: fn(nil: ref Draw->Context, nil: list of string);
171};
172
173init(nil: ref Draw->Context, nil: list of string)
174{
175 sys := load Sys Sys->PATH;
176 sys->print("Hello World!\n");
177}
178
179With this in a helloworld.b file, compiling it produces helloworld.dis. This contains the compiled bytecode for the Dis virtual machine. disdep will list the dependencies that it requires:
180
181; limbo helloworld.b
182; ./helloworld
183Hello World!
184; disdep helloworld.dis
185; disdep /dis/emuinit.dis
186/dis/lib/arg.dis
187/dis/sh.dis
188/dis/lib/bufio.dis
189/dis/lib/env.dis
190/dis/lib/readdir.dis
191/dis/lib/filepat.dis
192/dis/lib/string.dis
193
194This shows the helloworld has no dependencies outside of what's already built into emu and emuinit.dis contains a few. Creating a directory layout with just these files and emu should be enough to run helloworld. In the following example foo is the directory containing the full Inferno distribution where helloworld.b was compiled.
195
196$ mkdir -p hw/dis/lib
197$ cd hw
198$ cp ../foo/Linux/386/bin/emu .
199$ cp ../foo/dis/emuinit.dis dis/
200$ cp ../foo/dis/lib/arg.dis dis/lib/
201$ cp ../foo/dis/lib/bufio.dis dis/lib/
202$ cp ../foo/dis/lib/env.dis dis/lib/
203$ cp ../foo/dis/lib/readdir.dis dis/lib/
204$ cp ../foo/dis/lib/filepat.dis dis/lib/
205$ cp ../foo/dis/lib/string.dis dis/lib/
206$ cp ../foo/dis/sh.dis dis/
207$ cp ../foo/usr/inferno/helloworld.dis .
208$ ./emu -r. helloworld
209Hello World!
210
211Some of these dependencies aren't needed, for example sh.dis as we don't run the shell. See Powerman's post for a more extensive example that has dependencies.
212Bundling application into emu
213
214Instead of distributing a directory of files as in the previous example it's possible to bundle a root filesystem inside the emu program. This allows distributing just a single executable that runs the Limbo application. The steps to do this involve finding the dependencies of the program as above and creating a kernel configuration file that lists them.
215
216The file /emu/Linux/emu is the kernel configuration file for the Linux Inferno VM. It has a root section which defines the root filesystem:
217
218root
219 /dev /
220 /fd /
221 /prog /
222 /prof /
223 /net /
224 /net.alt /
225 /chan /
226 /nvfs /
227 /env /
228# /dis
229# /n
230# /icons
231# /osinit.dis
232# /dis/emuinit.dis
233# /dis/lib/auth.dis
234# /dis/lib/ssl.dis
235# /n/local /
236
237The actual filesystem as located by the -r command line argument to emu is overlayed on top of this. Copying this file and adding our dependencies will bundle them into a custom build executable. This is what the root section of our helloworld looks like:
238
239root
240 /dev /
241 /fd /
242 /prog /
243 /prof /
244 /net /
245 /net.alt /
246 /chan /
247 /nvfs /
248 /env /
249 /dis
250 /dis/emuinit.dis /usr/chris/helloworld.dis
251
252Here we make helloworld.dis appear in the root filesystem as /dis/emuinit.dis, which is the first program run as the system comes up. Note that this file must use tabs, not spaces, for the entries. An executable can be compiled with this bundled root filesystem with:
253
254$ cd emu/Linux
255$ ...create helloworld configuration file...
256$ mk CONF=helloworld
257
258This produces an executable o.helloworld which when run will execute the helloworld.dis embedded inside it:
259
260$ ./o.helloworld
261Hello World!
262
263When stripped the executable is about 1.5MB. This executable links to the X11 libraries. For a headless system you can remove the dependancy by using the emu-g kernel configuration file as a base. This removes the drivers that use X11 and prevents linking against the X11 libraries. The resulting executable when stripped is now 700K.
264
265The executable produced by this process has some dynamic dependencies - libc, etc - that most glibc applications have. It should be possible to use musl-libc to produce a static binary and I'll cover this in another post.
266Tags: inferno
267
2682016-07-18
269Borrowing in Pony
270
271The 'TL;DR' of this post on how to borrow internal fields of iso objects in Pony is:
272
273 To borrow fields internal to an iso object, recover the object to a ref (or other valid capability) perform the operations using the field, then consume the object back to an iso.
274
275Read on to find out why.
276
277In this post I use the term borrowing to describe the process of taking a pointer or reference internal to some object, using it, then returning it. An example from C would be something like:
278
279void* new_foo();
280void* get_bar(foo* f);
281void delete_foo(foo* f);
282
283...
284void* f = new_foo();
285void* b = get_bar(f);
286...
287delete_foo(f);
288
289Here a new foo is created and a pointer to a bar object returned from it. This pointer is to data internal to foo. It's important not to use it after foo is deleted as it will be a dangling pointer. While holding the bar pointer you have an alias to something internal to foo. This makes it difficult to share foo with other threads or reason about data races. The foo object could change the bar data without the holder of the borrowed pointer to bar knowing making it a dangling pointer, or invalid data, at any time. I go through a real world case of this in my article on using C in the ATS programming language.
290
291Pony has the concept of a reference to an object where only one pointer to that object exists. It can't be aliased and nothing else can read or write to that object but the current reference to it. This is the iso reference capability. Capabilities are 'deep' in pony, rather than 'shallow'. This means that the reference capability of an alias to an object affects the reference capabilities of fields of that object as seen by that alias. The description of this is in the viewpoint adaption section of the Pony tutorial.
292
293The following is a Pony equivalent of the previous C example:
294
295class Foo
296 let bar: Bar ref
297...
298let f: Foo ref = Foo.create()
299let b: Bar ref = f.bar
300
301The reference capability of f determines the reference capability of bar as seen by f. In this case f is a ref (the default of class objects) which according to the viewpoint adaption table means that bar as seen by f is also a ref. Intuitively this makes sense - a ref signifies multiple read/write aliases can exist therefore getting a read/write alias to something internal to the object is no issue. A ref is not sendable so cannot be accessed from multiple threads.
302
303If f is an iso then things change:
304
305class Foo
306 let bar: Bar ref
307...
308let f: Foo iso = recover iso Foo.create() end
309let b: Bar tag = f.bar
310
311Now bar as seen by f is a tag. A tag can be aliased but cannot be used to read/write to it. Only object identity and calling behaviours is allowed. Again this is intuitive. If we have a non-aliasable reference to an object (f being iso here) then we can't alias internally to the object either. Doing so would mean that the object could be changed on one thread and the internals modified on another giving a data race.
312
313The viewpoint adaption table shows that given an iso f it's very difficult to get a bar that you can write to. The following read only access to bar is ok:
314
315class Foo
316 let bar: Bar val
317...
318let f: Foo iso = recover iso Foo.create() end
319let b: Bar val = f.bar
320
321Here bar is a val. This allows multiple aliases, sendable across threads, but only read access is provided. Nothing can write to it. According to viewpoint adaption, bar as seen by f is a val. It makes sense that given a non-aliasable reference to an object, anything within that object that is immutable is safe to borrow since it cannot be changed. What if bar is itself an iso?
322
323class Foo
324 let bar: Bar iso = recover iso Bar end
325...
326let f: Foo iso = recover iso Foo.create() end
327let b: Bar iso = f.bar
328
329This won't compile. Viewpoint adaption shows that bar as seen by f is an iso. The assignment to b doesn't typecheck because it's aliasing an iso and iso reference capabilities don't allow aliasing. The usual solution when a field isn't involved is to consume the original but it won't work here. The contents of an objects field can't be consumed because it would then be left in an undefined state. A Foo object that doesn't have a valid bar is not really a Foo. To get access to bar externally from Foo the destructive read syntax is required:
330
331class Foo
332 var bar: Bar iso = recover iso Bar end
333...
334let f: Foo iso = recover iso Foo.create() end
335let b: Bar iso = f.bar = recover iso Bar end
336
337This results in f.bar being set to a new instance of Bar so it's never in an undefined state. The old value of f.bar is then assigned to b. This is safe as there are no aliases to it anymore due to the first part of the assignment being done first.
338
339What if the internal field is a ref and we really want to access it as a ref? This is possible using recover. As described in the tutorial, one of the uses for recover is:
340
341 "Extract" a mutable field from an iso and return it as an iso.
342
343This looks like:
344
345class Foo
346 let bar: Bar ref
347...
348let f: Foo iso = recover iso Foo end
349let f' = recover iso
350 let f'': Foo ref = consume f
351 let b: Bar ref = f''.bar
352 consume f''
353 end
354
355Inside the recover block f is consumed and returned as a ref. The f alias to the object no longer exists at this point and we have the same object but as a ref capability in f''. bar as seen by f'' is a ref according to viewpoint adaption and can now be used within the recover block as a ref. When the recover block ends the f'' alias is consumed and returned out of the block as an iso again in f'.
356
357This works because inside the recover block only sendable values from the enclosing scope can be accessed (ie. val, iso, or tag). When exiting the block all aliases except for the object being returned are destroyed. There can be many aliases to bar within the block but none of them can leak out. Multiple aliases to f' can be created also and they are not going to leaked either. At the end of the block only one can be returned and by consuming it the compiler knows that there are no more aliases to it so it is safe to make it an iso.
358
359To show how the ref aliases created within the recover block can't escape, here's an example of an erroneous attempt to assign the f' alias to an object in the outer scope:
360
361class Baz
362 var a: (Foo ref | None) = None
363 var b: (Foo ref | None) = None
364
365 fun ref set(x: Foo ref) =>
366 a = x
367 b = x
368
369class Bar
370
371class Foo
372 let bar: Bar ref = Bar
373
374var baz: Baz iso = recover iso Baz end
375var f: Foo iso = recover iso Foo end
376f = recover iso
377 let f': Foo ref = consume f
378 baz.set(f')
379 let b: Bar ref = f'.bar
380 consume f'
381 end
382
383If this were to compile then baz would contain two references to the f' object which is then consumed as an iso. f would contain what it thinks is non-aliasable reference but baz would actually hold two additional references to it. This fails to compile at this line:
384
385main.pony:20:18: receiver type is not a subtype of target type
386 baz.set(f')
387 ^
388Info:
389main.pony:20:11: receiver type: Baz iso!
390 baz.set(f')
391 ^
392main.pony:5:3: target type: Baz ref
393 fun ref set(x: Foo ref) =>
394 ^
395main.pony:20:18: this would be possible if the arguments and return value were all sendable
396 baz.set(f')
397 ^
398
399baz is an iso so is allowed to be accessed from within the recover block. But the set method on it expects a ref receiver. This doesn't work because the receiver of a method of an object is also an implicit argument to that method and therefore needs to be aliased. In this way it's not possible to store data created within the recover block in something passed into the recover block externally. No aliases can be leaked and the compiler can track things easily.
400
401There is something called automatic receiver recovery that is alluded to in the error message ("this would be possible...") which states that if the arguments were sendable then it is possible for the compiler to work out that it's ok to call a ref method on an iso object. Our ref arguments are not sendable which is why this doesn't kick in.
402
403A real world example of where all this comes up is using the Pony net/http package. A user on IRC posted the following code snippet:
404
405use "net/http"
406class MyRequestHandler is RequestHandler
407
408 let env: Env
409
410 new val create(env': Env) =>
411 env = env'
412
413 fun val apply(request: Payload iso): Any =>
414 for (k, v) in request.headers().pairs() do
415 env.out.print(k)
416 env.out.print(v)
417 end
418
419 let r = Payload.response(200)
420 r.add_chunk("Woot")
421 (consume request).respond(consume r)
422
423The code attempts to iterate over the HTTP request headers and print them out. It fails in the request.headers().pairs() call, complaining that tag is not a subtype of box in the result of headers() when calling pairs(). Looking at the Payload class definition shows:
424
425class iso Payload
426 let _headers: Map[String, String] = _headers.create()
427
428 fun headers(): this->Map[String, String] =>
429 _headers
430
431In the example code request is an iso and the headers function is a box (the default for fun). The return value of headers uses an arrow type. It reads as "return a Map[String, String] with the reference capability of _headers as seen by this". In this example this is the request object which is iso. _headers is a ref according to the class definition. So it's returning a ref as seen by an iso which according to viewpoint adaption is a tag.
432
433This makes sense as we're getting a reference to the internal field of an iso object. As explained previously this must be a tag to prevent data races. This means that pairs() can't be called on the result as tag doesn't allow function calls. pairs() is a box method which is why the error message refers to tag not being a subtype of box.
434
435To borrow the headers correctly we can use the approach done earlier of using a recover block:
436
437fun val apply(request: Payload iso): Any =>
438 let request'' = recover iso
439 let request': Payload ref = consume request
440 for (k, v) in request'.headers().pairs() do
441 env.out.print(k)
442 env.out.print(v)
443 end
444 consume request'
445 end
446 let r = Payload.response(200)
447 r.add_chunk("Woot")
448 (consume request'').respond(consume r)
449
450In short, to borrow fields internal to an iso object, recover the object to a ref (or other valid capability) perform the operations using the field, then consume the object back to an iso.
451Tags: pony
452
4532016-07-14
454Concurrency in Wasp Lisp
455
456Wasp Lisp has a light weight co-operative threading model that's allows programming in an Actor style. It's possible to serialize Wasp values and send them to other processes and machines to be deserialized and run. MOSREF uses this to compile Lisp code on the console process and send the bytecode to drone processes to execute. This allows drones to operate without the Lisp compiler present.
457Spawning threads
458
459Threads are created using the spawn function. It takes the function to run as a thread as an argument:
460
461(spawn (lambda () (print "Hello World\n")))
462
463Communication between threads is done using queues. A queue is an unbounded channel that can have many senders but only one receiver. The function send adds data to the queue and wait receives data. If there is no data in the queue then wait blocks. Input/Output in Wasp Lisp is done using the same wait/send mechanism making it easy to pipeline data from console and file output to sockets.
464Implementing Actors
465
466A basic Actor can be implemented like the following:
467
468(define (actor1)
469 (define counter 0)
470 (define chan (make-queue))
471
472 (define (loop)
473 (define msg (wait chan))
474 (cond
475 ((eq? msg 'inc)
476 (set! counter (+ 1 counter)))
477 ((eq? msg 'dec)
478 (set! counter (- 1 counter)))
479 ((and (list? msg) (eq? (car msg) 'get))
480 (send counter (cadr msg))))
481 (loop))
482
483 (spawn loop)
484 chan)
485
486actor1 is a function that contains a counter holding an numeric value. It creates chan, a queue for holding messages, spawns a thread to run loop and returns the chan so messages can be queued for loop to process.
487
488loop waits for a message on chan. This is a blocking call and the thread will go idle until a message is queued. It processes the message, incrementing or decrementing the counter as requested. An additional message, get, can be used to get the value of the counter. That message also includes a channel object to place the result in. loop recursively calls itself to continue.
489
490A sample interaction is:
491
492>> (define a1 (actor1))
493>> (define result (make-queue))
494>> (send (list 'get result) a1)
495>> (wait result)
496:: 0
497>> (send 'inc a1)
498>> (send (list 'get result) a1)
499>> (wait result)
500:: 1
501
502This creates an actor and a queue to receive results. It asks for the current value of the actor, increments it, then asks again.
503Updating an Actor
504
505It's possible to update the code for an Actor without stopping the application. Running in a Lisp REPL means you can change functions on the fly but you can't change the internal implementation of a running loop from the REPL if that loop is internal to a function. A way around this is to provide the Actor with the means to receive a function as a message that performs the update. Here is an example of an updatable actor:
506
507(define (actor3)
508 (define counter 0)
509 (define chan (make-queue))
510
511 (define (loop chan)
512 (define msg (wait chan))
513 (cond
514 ((eq? msg 'inc)
515 (set! counter (+ 1 counter)))
516 ((eq? msg 'dec)
517 (set! counter (- 1 counter)))
518 ((and (list? msg) (eq? (car msg) 'get))
519 (send counter (cadr msg)))
520 ((function? msg)
521 (return ((msg counter) chan))))
522 (loop chan))
523
524 (spawn loop chan)
525 chan)
526
527This code contains an additional branch in the cond to check if the message is a function. If it is then that function is called passing the current value of the counter. It is expected to return a function which will be the new loop to call. This can contain any code and effectively updates the entire actor with new functionality. An example update function to change the messages to increment/decrement by two is:
528
529(define (update oldstate)
530 (define counter (* oldstate 2))
531 (define (loop chan)
532 (define msg (wait chan))
533 (cond
534 ((eq? msg 'inc)
535 (set! counter (+ 2 counter)))
536 ((eq? msg 'dec)
537 (set! counter (- 2 counter)))
538 ((and (list? msg) (eq? (car msg) 'get))
539 (send counter (cadr msg)))
540 ((function? msg)
541 (return ((msg counter) chan))))
542 (loop chan))
543 loop)
544
545An example interaction of the actor and upgrading it is:
546
547>> (define a3 (actor3))
548>> (define result (make-queue))
549>> (send 'inc a3)
550>> (send (list 'get result) a3)
551>> (wait result)
552:: 1
553>> (send update a3) ;; Updating the actor here
554>> (send (list 'get result) a3)
555>> (wait result)
556:: 2 ;; This shows the new counter value that 'update' changed
557>> (send 'inc a3)
558>> (send (list 'get result) a3)
559>> (wait result)
560:: 4 ;; Amount is now incrementing by two
561>> (send 'inc a3)
562>> (send (list 'get result) a3)
563>> (wait result)
564:: 6
565
566This is a variant of Joe Armstrong's Erlang Universal Server allowing a server to be updated to do anything.
567Filters
568
569An idiom when programming in an Actor or coroutine style is to write small processes that take an input, modify it in some way, and send it to another process to do something else. A program becomes a chain or pipeline of these individual processes. Wasp Lisp calls these small units of functionality filters. They are described in filter.ms as:
570
571 A process that waits for data from an input channel, and sends data to an output channel. Filters are constructed using a constructor function, then wired together using either the input-chain or output-chain functions."
572
573This is an example of a line filter from the Wasp source code;
574
575(define-filter (line-filter)
576 (define buf (make-string 80))
577
578 (define (parse)
579 (forever
580 (define next (string-read-line! buf))
581 (if next (send next out)
582 (return))))
583
584 (define (line-loop)
585 (forever
586 (define next (wait-input in))
587 (cond
588 ((string? next)
589 (string-append! buf next)
590 (parse))
591 ((eq? next 'close)
592 (return))
593 (else
594 (send-output next out)))))
595
596 (line-loop)
597
598 (send-output buf out)
599 (send-output 'close out))
600
601A line-filter receives strings of bytes on the input channel and outputs a complete line on the output channel when it has one. It does this by appending received bytes onto a string buffer and checking if that buffer contains a line. If it does it removes the line data from the buffer and sends it to the output channel. It then continues to wait for data on the input channel. An example of usage:
602
603>> (import "lib/filter")
604>> (import "lib/line-filter")
605>> (define q (make-queue))
606>> (define lines (input-chain q (line-filter)))
607>> (spawn (lambda () (forever (print (wait lines)))))
608
609>> (send "hello" q)
610>> (send "world\n" q)
611helloworld
612>> (send "foo\nbar" q)
613foo
614>> (send "baz\n" q)
615barbaz
616
617This creates a queue, q for input data. It creates a chain containing only one filter, the line-filter. It returns the output channel which contains the filtered data. Data placed in q is retrieved by the line filter and when a line is received it is sent to the output channel. A thread is spawned to loop forever printing any lines from the output channel. Notice in the manual sending of data to the channel q that output is only printed by the spawned thread when a line is completed.
618
619Wasp Lisp comes with some default filters for parsing s-expressions, encrypting and decrypting data and fuzzing data amongst other things. Scott Dunlop wrote about coroutines and filters on the Wasp blog.
620Sending data to other OS processes
621
622Some Wasp values can be serialized and deserialized. This provides a way to send values to other wasp instances running in different OS processes or machines. Lisp objects are serialized using freeze and unserialized using thaw.
623
624The following server function starts a TCP server on port 10000. Clients connnected to it send Lisp objects to it and it prints it to the standard output on the server process.
625
626(import "lib/tcp-server")
627
628(define (server)
629 (define server-output (current-output))
630
631 (define (acceptor)
632 (forever
633 (define data (wait))
634 (with-output server-output
635 (print (format (thaw data)))
636 (print "\n"))))
637
638 (spawn-tcp-server 10000 acceptor))
639
640The acceptor function is called with its current input and output bound to the TCP stream. For this reason we capture the value of current-output before it is bound so we can output to the server console rather than to the TCP stream. A sample test:
641
642;; On server
643>> (server)
644
645;; On client
646>> (define s (tcp-connect "127.0.0.1" 10000))
647>> (send (freeze "foo") s)
648
649;; On Server
650"foo"
651
652;; On Client
653>> (send (freeze 66) s)
654
655;; On Server
65666
657
658;; On Client
659>> (send (freeze '(one (two three))) s)
660
661;; On Server
662(one (two three))
663
664Notice that all i/o is done using the 'send' and 'wait' channel operators. This means we can use a filter to do the freezing/thawing automatically and Wasp has a freeze-filter and thaw-filter that does this. The server becomes:
665
666(import "lib/tcp-server")
667(import "lib/package-filter")
668(import "lib/filter")
669(import "lib/format-filter")
670
671(define (server2)
672 (define server-output (current-output))
673
674 (define (acceptor)
675 (define chan (input-chain (current-input)
676 (thaw-filter)
677 (format-filter)))
678 (forever
679 (define data (wait chan))
680 (print* data "\n")))
681
682 (spawn-tcp-server 10000 acceptor))
683
684Usage from a client is:
685
686>> (import "lib/filter")
687>> (import "lib/package-filter")
688
689>> (define s (tcp-connect "127.0.0.1" 10000))
690>> (define chan (output-chain s (freeze-filter)))
691>> (send "hello" chan)
692>> (send '(one (two three)) chan)
693
694Through the use of the thaw/freeze filter there is no need to manually call freeze and thaw.
695Sending bytecode to other processes
696
697Unfortunately it's not possible to freeze or thaw closures or functions. It is possible however to assemble Lisp to bytecode and send that. This enables sending new functions across OS processes and is how MOSREF is able to compile Lisp on the console and send it to the drone. This example will compile a function from source to bytecode and run it:
698
699>> (define code '((print "Hello World\n")))
700>> (define proc (assemble (optimize (compile code))))
701>> (proc)
702Hello World
703
704The result of assemble can be frozen, sent somewhere and thawed:
705
706>> (define x (freeze (assemble (optimize (compile '((print "Hello World\n")))))))
707>> (define y (thaw x))
708>> (y)
709Hello World
710
711Using this we can have an upgradable server process:
712
713(define (server3)
714 (define server-output (current-output))
715
716 (define (acceptor)
717 (define chan (input-chain (current-input)
718 (thaw-filter)))
719
720 (define (loop chan)
721 (define data (wait chan))
722 (cond
723 ((function? data)
724 (return ((data) chan)))
725 (else
726 (print* "OLD: " (format data) "\n")
727 (return (loop chan)))))
728 (loop chan))
729
730 (spawn-tcp-server 10000 acceptor))
731
732This will display the data sent to the server prefixed by "OLD:" unless it is sent a function. In which case it calls that function as the new server loop. An upgraded server loop to prefix with "NEW: " is:
733
734(define (new-server3)
735 (assemble
736 (optimize
737 (compile
738 '((define (loop chan)
739 (define data (wait chan))
740 (cond
741 ((function? data)
742 (return ((data) chan)))
743 (else
744 (print* "NEW: " (format data) "\n")
745 (return (loop chan))))))))))
746
747We can't send a function directly so this compiles the new loop from source and returns the compiled procedure. This can be frozen, sent to the server and it will execute it as the new loop. An example interaction:
748
749;; On Server
750>> (server3)
751
752;; On Client
753>> (define s (tcp-connect "127.0.0.1" 10000))
754>> (define chan (output-chain s (freeze-filter)))
755>> (send '(one (two three)) chan)
756
757;; On Server
758OLD: (one (two three))
759
760;; On Client
761>> (send (new-server3) chan)
762>> (send '(one (two three)) chan)
763
764;; On Server
765>> NEW: (one (two three))
766
767Why not send the source to the server process and have it eval it? The approach of sending the bytecode allows the server process to skip including the Lisp compiler. The Wasp VM includes an interpreter and deserializer - the compiler and other libraries are all in Lisp. A Wasp executable consists of the VM stub with bytecode appended to the end of it. On execution it looks for the bytecode, deserializes it and runs it. This provides a minimal program that can have functionality added by sending it bytecode as needed.
768An aside on tail call optimization
769
770It's important that a process loop is tail recursive otherwise each call through the loop will increase stack size and eventually exhaust memory. The following is not tail recursive in Wasp Lisp, even though it looks like it should be:
771
772(define (test1 chan)
773 (define msg (wait chan))
774 (cond
775 ((eq msg 'foo)
776 (test1 chan))
777 ((eq msg 'bar)
778 (test1 chan))
779 (else
780 (test1 chan))))
781
782This is because the recursive call to 'test1' compiles down to bytecode that looks like:
783
784(newf)
785(ldg eq)
786(arg)
787(ldg msg)
788(arg)
789(ldc bar)
790(arg)
791(call)
792(jf false-47) ;; If the msg is not 'bar then jump to false-47
793...
794false-47
795(newf)
796(ldg test1)
797(arg)
798(ldg chan)
799(arg)
800(call) ;; recursively call 'test1'
801done-46
802done-44
803(retn) ;; return from function 'test1'
804
805The stack frame for test1 is not exited (the retn instruction) until after the recursive call is done. Compare this to the obvious tail recursive case:
806
807(newf)
808(ldg wait)
809(arg)
810(ldg chan)
811(arg)
812(call)
813(stg msg)
814(newf)
815(ldg test2)
816(arg)
817(ldg chan)
818(arg)
819(tail)
820
821Note that tail instruction. This does an immediate jump rather than a call so a retn is not necessary. The call stack does not grow. The difference between the two cases is due to the way the Wasp Lisp compiler generates the instructions and optimizes looking for tail calls. The instructions generated can be viewed using:
822
823(define x '(define (test2 chan)
824 (define msg (wait chan))
825 (test2 chan)))
826(define code (compile x))
827(for-each (lambda (x) (print* (format x) "\n")) code)
828
829Using compile shows the first pass which does not look for tail calls:
830
831(newf)
832(ldg test2)
833(arg)
834(ldg chan)
835(arg)
836(call)
837(retn)
838
839Notice the call followed by retn. This is the sequence that optimize looks for to generate the tail instruction:
840
841(define x '(define (test2 chan)
842 (define msg (wait chan))
843 (test2 chan)))
844(define code (optimize (compile x)))
845(for-each (lambda (x) (print* (format x) "\n")) code)
846...
847(newf)
848(ldg test2)
849(arg)
850(ldg chan)
851(arg)
852(tail)
853
854Looking back at the instructions for test1 the call is followed by a jump or a label before retn so the optimizer misses it. This can be worked around by doing an explicit return statement:
855
856(define (test3 chan)
857 (define msg (wait chan))
858 (cond
859 ((eq msg 'foo)
860 (return (test1 chan)))
861 ((eq msg 'bar)
862 (return (test1 chan)))
863 (else
864 (return (test1 chan)))))
865
866The code in the cond branches generates to the following which is now a tail call:
867
868(jf false-93)
869(newf)
870(ldg test1)
871(arg)
872(ldg chan)
873(arg)
874(tail)
875
876Some things to note
877
878The Wasp VM is single threaded and non-preemptive. Threads yield to the scheduler explicitly using yield or implicitly when doing i/o or waiting on a queue. The bytecode is cross platform. Serialized objects on one architecture can be deserialized on another. The Wasp VM history comes from Mosquito Lisp and MOSREF - a penetration testing platform. It's written in C with some GNU extensions (nested functions are used in the VM).
879
880This post came about from exploring the difference in Actor programming in the Pony programming language and a dynamic language where the Actor model isn't explicit. The programming style is similar in that pipelines of calls to actors to transform data is a common idiom.
881
882Wasp Lisp isn't actively developed anymore but the author, Scott Dunlop, still processes pull requests and monitors it. I like to use it for projects and tinker with it as it's an interesting little cross platform lisp. MOSREF is useful as a way to access and maintain servers of different architectures, aside from its use as a penetration testing tool.
883
884Some other Wasp resources:
885
886 Wasp Lisp main site
887 Wasp Developments Blog
888 Source on Github
889 Ephemeral Security Mosquito Lisp on the Internet Archive Wayback Machine.
890
891Tags: waspvm
892
8932016-06-05
894Building Static Wasp Lisp Binaries on Linux
895
896Wasp Lisp builds binaries that are linked dynamically to glibc. This ties the binary to specific versions of Linux. It's usually not possible to run on an OS with older glibc versions than what it was compiled against. I wanted to be able to run a single binary of Wasp Lisp and MOSREF drones on new Ubuntu versions and some machines with an older version of Ubuntu. To do this I needed to have the libc linked statically.
897
898Changing Wasp Lisp to statically link glibc doesn't work though. Some networking routines in glibc require dynamic linking. If glibc is statically linked then networking doesn't work.
899
900The solution I opted for is to use musl libc instead of glibc. This is a libc that was designed to be statically linked. To buid Wasp Lisp binaries with musl it required:
901
902 Building musl libc
903 Building libevent using musl libc headers
904 Building Wasp Lisp against musl and libevent
905
906Building musl libc
907
908Building musl libc requires using git to clone the repository and following the standard configure, make, make install invocations. The bin directory for the musl tools is added to the PATH:
909
910$ git clone git://git.musl-libc.org/musl
911$ cd musl
912$ ./configure
913$ make
914$ sudo make install
915$ export PATH=$PATH:/usr/local/musl/bin/
916
917Building libevent
918
919Building libevent with musl requires using the musl-gcc command which was installed by the previous step. This invokes GCC with the required options to use musl. The following steps performs the build:
920
921$ wget https://github.com/libevent/libevent/releases/download/release-2.0.22-stable/libevent-2.0.22-stable.tar.gz
922$ tar xvf libevent-2.0.22-stable.tar.gz
923$ cd libevent-2.0.22-stable/
924$ ./configure --prefix=/tmp/musl/usr CC=musl-gcc --enable-static --disable-shared
925$ make
926$ make install
927
928Building Wasp Lisp
929
930The Wasp VM source requires a change to the Makefile.cf to use static linking for all libraries. This changes:
931
932EXEFLAGS += -Wl,-Bstatic $(STATICLIBS) -Wl,-Bdynamic $(DYNAMICLIBS)
933
934to:
935
936EXEFLAGS += -static $(STATICLIBS) $(DYNAMICLIBS)
937
938I've made this change in the static branch of my github fork . This branch also includes some other changes from the official repository for real number support. Building with musl and libevent is done with:
939
940$ git clone https://github.com/doublec/WaspVM --branch static
941$ cd WaspVM
942$ CC=musl-gcc CFLAGS="-I /tmp/musl/usr/include -L /tmp/musl/usr/lib" make repl
943
944This runs directly into the Lisp REPL. The following confirms a static binary:
945
946$ ldd wasp
947not a dynamic executable
948
949Building MOSREF
950
951The stub generated is also static and can be used to build static drones:
952
953$ cd mod
954$ ../waspc -exe ../mosref bin/mosref
955$ chmod +x ../mosref
956$ ../mosref
957console> set addr=xx.xx.xx.xx
958console> set port=8000
959console> drone mydrone foo linux-x86_64
960Drone executable created.
961
962The generated drone should run on a wider range of Linux versions than the non-static build at the cost of a larger size. I rename the waspvm-linux-x86_64 stub to be waspvm-musl-x86-64 so I can generate static drones or dynamic linked drones as needed from the MOSREF console by using linux-x86_64 or musl-x86_64 respectively.