I recently put together a short demo about Q learning and SARSA learning for a class I was taking. The demo involved running the two algorithms in parallel in a simple maze world. The world, the agent, and the learned Q function are drawn using OpenGL.
A few people suggested I make the presentation and demo code available for others, so here it is: You can download the code for the reinforcement learning demo from GitHub, and I have attached the presentation as well.
Over the holiday, I’ve added a few tools to the NXT Tools I’ve been developing. One is a simple python tool to use gstreamer to convert any gstreamer readable audio format to the NXT’s native RSO format. Also I’ve added a very basic tool to use a computer to interface a Wii remote (aka wiimote) to the NXT brick.
I’ve been recently playing with my new Lego NXT robotics kit. Being a Linux nerd, I naturally wanted to do the programming on Linux using languages I like. To this end, I’ve been using the nxt-python project and NXC. However, I found the nxt-push script a little cumbersome and not very user friendly, and the nxt-filer gui was less than useful. I didn’t want to develop a complete file manager interface. After all, file managers have been redesigned and reimplemented a million different ways. All I really needed was a file interface to the NXT. Enter Fuse. So, I spent some time yesterday and developed a Fuse interface for the NXT using nxt-python and fuse python bindings. You can find the interface in, what will be, a collection of tools for manipulating and controlling the NXT brick on GitHub.
After having used CVS, Subversion, and Perforce, I’ve recently been using Git for all of my version control. The two things I like about above the others mentioned are its speed and trivial and intuitive branching. Though I can’t say I was ever a fan of CVS branches, I always found that the subversion/perforce approach of just creating another directory when you want to branch to somewhat ignore the semantics of being a branch. Why shouldn’t every version be a new directory? Obviously that would get cluttered and confusing. Why would you not expect the same of branches? Yes, I understand that directory copying is cheap in Subversion and Perforce, and I understand they do preserve history, but I expect branches to split and merge and though directories do represent the splitting very well, they unfortunately don’t represent the concept of a merge very well. Git on the otherhand, has explicit branches that are separate from the concept of a directory. Plus, it has many powerful tools for managing those branches, not to mention GitHub.