Josh Bryan

Reinforcement Learning Demo

jbryan — Fri, 17 Dec 2010 05:40:48 +0000

I recently put together a short demo about Q learning and SARSA learning for a class I was taking. The demo involved running the two algorithms in parallel in a simple maze world. The world, the agent, and the learned Q function are drawn using OpenGL.

A few people suggested I make the presentation and demo code available for others, so here it is: You can download the code for the reinforcement learning demo from GitHub, and I have attached the presentation as well.

More NXT Tools

jbryan — Tue, 05 Jan 2010 03:25:00 +0000

Over the holiday, I’ve added a few tools to the NXT Tools I’ve been developing. One is a simple python tool to use gstreamer to convert any gstreamer readable audio format to the NXT’s native RSO format. Also I’ve added a very basic tool to use a computer to interface a Wii remote (aka wiimote) to the NXT brick.

Fuse + Python + Lego NXT

jbryan — Tue, 15 Dec 2009 20:18:18 +0000

I’ve been recently playing with my new Lego NXT robotics kit. Being a Linux nerd, I naturally wanted to do the programming on Linux using languages I like. To this end, I’ve been using the nxt-python project and NXC. However, I found the nxt-push script a little cumbersome and not very user friendly, and the nxt-filer gui was less than useful. I didn’t want to develop a complete file manager interface. After all, file managers have been redesigned and reimplemented a million different ways. All I really needed was a file interface to the NXT. Enter Fuse. So, I spent some time yesterday and developed a Fuse interface for the NXT using nxt-python and fuse python bindings. You can find the interface in, what will be, a collection of tools for manipulating and controlling the NXT brick on GitHub.

Git … my favorite vcs

jbryan — Sat, 14 Nov 2009 19:50:30 +0000

After having used CVS, Subversion, and Perforce, I’ve recently been using Git for all of my version control. The two things I like about above the others mentioned are its speed and trivial and intuitive branching. Though I can’t say I was ever a fan of CVS branches, I always found that the subversion/perforce approach of just creating another directory when you want to branch to somewhat ignore the semantics of being a branch. Why shouldn’t every version be a new directory? Obviously that would get cluttered and confusing. Why would you not expect the same of branches? Yes, I understand that directory copying is cheap in Subversion and Perforce, and I understand they do preserve history, but I expect branches to split and merge and though directories do represent the splitting very well, they unfortunately don’t represent the concept of a merge very well. Git on the otherhand, has explicit branches that are separate from the concept of a directory. Plus, it has many powerful tools for managing those branches, not to mention GitHub.

Life Update

jbryan — Sat, 14 Nov 2009 19:38:28 +0000

I’ve been a little busy … well, very busy … this last year, so here is a little update about projects in my life. I finished up my masters thesis this last semester, and started work toward my Ph.D. in computer science at UIC. I am researching rationality in multiagent settings for the UIC Multi Agent Systems laboratory. Particularly, I am looking at ways to update POMDP models from agent experience. Between school, various freelance projects, and a short year long career at CashNetUSA, I’ve been pretty busy and haven’t been posting much, hopefully that will change.

Colored man pages

jbryan — Fri, 05 Sep 2008 20:00:58 +0000

I just discovered this very cool tip. If you want a colored man pages, adding the following to your environment does it:

export LESS_TERMCAP_mb=$'\E[01;31m' export LESS_TERMCAP_md=$'\E[01;31m' export LESS_TERMCAP_me=$'\E[0m' export LESS_TERMCAP_se=$'\E[0m' export LESS_TERMCAP_so=$'\E[01;44;33m' export LESS_TERMCAP_ue=$'\E[0m' export LESS_TERMCAP_us=$'\E[01;32m'

I found the tip while searching for a way to preserve the colored output of grep after it has been piped into less. ‘less -r’ preserves the terminal escape sequences from the input. The following will get preserve the colors:

grep --color=always | less -r

Origin of Wealth

jbryan — Sat, 16 Aug 2008 21:21:28 +0000

Origin of Wealth is a very entertaining and captivating survey of recent developments in economics. He repeatedly takes traditional economic theories to task for failing to match real world empirical data while painting a picture of the economy as a complex, adaptive, and evolutionary system rather than the traditional overly simplistic system of market equilibrium. This is definitely worthwhile read.

Back in town ….

jbryan — Fri, 15 Aug 2008 23:00:18 +0000

I just returned from a month long vacation in Europe. My wife and I took a budget (hostels and Easy Jet) tour of Europe starting in Ireland, and moving through Sweden, Norway, Denmark, Holland, Germany, and England. The trip was a blast, but I’m still glad to be back home. You can see pictures of our trip on Jen’s Picasa album.

While on the trip, I decided to spend more time maintaining this blog. We will see how strong my resolve to do so really is in the next few months. However, since I am paying for the blog, I may as well make more regular use of it. In addition, I think the writing could prove to be therapeutic.

LVM2 Mirrors vs. MD Raid 1

jbryan — Wed, 02 Jan 2008 23:19:51 +0000

I recently added a new hard drive to my desktop and reorganized the partition structure. In doing so I wanted my /home partition to be mirrored for safety, but I also wanted to use lvm to make managing my multiple partitions less painful. In my research, I found there are basically two ways to do this. One is to create a linux RAID1 device from the two drives and then use that device as a physical extent in a lvm volume group (here is a basic tutorial). The other method is to us LVM2′s mirroring capabilities. I personally would rather use the latter solution if there is no performance impact since moving the partitions around later would be much easier, however what I read gave the impression that LVM2 mirroring would be slower.

To test this, I setup a quick informal benchmark. I created 2 partitions on each of two drives and set one up as plain raid1 (no lvm) and the other as lvm with a mirror. I then ran bonnie++ on each of the partitions. The results were surprising. MD RAID1 gave about 25MB/sec write and 50MB/sec read. However LVM2 mirrored gave about 50MB/sec write and 55MB/sec read. Since the test was not perfectly sterile (there were other processes running at the same time) I would be willing to give these number +/- 10MB/sec. However still, LVM write was considerably faster than MD RAID1. I so far have no explanation for this, but will test further. If anyone has an explanation, I’d be happy to hear.

FEMA fake news

jbryan — Tue, 30 Oct 2007 04:27:04 +0000

I mean really, of all the juvenile things…