I recently put together a short demo about Q learning and SARSA learning for a class I was taking. The demo involved running the two algorithms in parallel in a simple maze world. The world, the agent, and the learned Q function are drawn using OpenGL.
A few people suggested I make the presentation and demo code available for others, so here it is: You can download the code for the reinforcement learning demo from GitHub, and I have attached the presentation as well.
Over the holiday, I’ve added a few tools to the NXT Tools I’ve been developing. One is a simple python tool to use gstreamer to convert any gstreamer readable audio format to the NXT’s native RSO format. Also I’ve added a very basic tool to use a computer to interface a Wii remote (aka wiimote) to the NXT brick.
I’ve been recently playing with my new Lego NXT robotics kit. Being a Linux nerd, I naturally wanted to do the programming on Linux using languages I like. To this end, I’ve been using the nxt-python project and NXC. However, I found the nxt-push script a little cumbersome and not very user friendly, and the nxt-filer gui was less than useful. I didn’t want to develop a complete file manager interface. After all, file managers have been redesigned and reimplemented a million different ways. All I really needed was a file interface to the NXT. Enter Fuse. So, I spent some time yesterday and developed a Fuse interface for the NXT using nxt-python and fuse python bindings. You can find the interface in, what will be, a collection of tools for manipulating and controlling the NXT brick on GitHub.
I recently added a new hard drive to my desktop and reorganized the partition structure. In doing so I wanted my /home partition to be mirrored for safety, but I also wanted to use lvm to make managing my multiple partitions less painful. In my research, I found there are basically two ways to do this. One is to create a linux RAID1 device from the two drives and then use that device as a physical extent in a lvm volume group (here is a basic tutorial). The other method is to us LVM2′s mirroring capabilities. I personally would rather use the latter solution if there is no performance impact since moving the partitions around later would be much easier, however what I read gave the impression that LVM2 mirroring would be slower.
To test this, I setup a quick informal benchmark. I created 2 partitions on each of two drives and set one up as plain raid1 (no lvm) and the other as lvm with a mirror. I then ran bonnie++ on each of the partitions. The results were surprising. MD RAID1 gave about 25MB/sec write and 50MB/sec read. However LVM2 mirrored gave about 50MB/sec write and 55MB/sec read. Since the test was not perfectly sterile (there were other processes running at the same time) I would be willing to give these number +/- 10MB/sec. However still, LVM write was considerably faster than MD RAID1. I so far have no explanation for this, but will test further. If anyone has an explanation, I’d be happy to hear.