Book Meme

Time for a little bit of nonsense.

I was following the news that Pownce is dead and started reading some of Leah Culver’s blog entries, which I usually try to check in every so often but don’t have in my feed for some reason. Loh and behold I found that she’s been playing with CouchDB too!

It’s always rewarding to know that someone who’s made such a professional impact is exploring things the same as I am (although I’m sure her learning curve is faster then mine ;> ).

In one of her posts I stumbled across the “book meme“. I don’t really listen to music like I should or enjoy a lot of the “fill in this questionnaire” memes that go around, but books… now that I can get into!

So here’s mine;

“If the yeast is active, it will have at least doubled its volume with a frothy head.” – Caveman Chemistry by Kevin M Dunn.


Book meme:

- Grab the nearest book.
- Open it to page 56.
- Find the fifth sentence.
- Post the text of the sentence in your journal along with these instructions.
- Don’t dig for your favorite book, the cool book, or the intellectual one: pick the CLOSEST.

Posted in uncategorized | 2 Comments

Training Neural Nets with CouchDB – part 1

My goal for this post is a bit technical and I’ll try warping both an artificial neural net, as well as my biological one, around an exploration of CouchDB, so read on if appropriate to your interests.

As you may have noticed in some of my earlier posts I’ve been playing with couchdb but it wasn’t until I started following Janl and reading some of his blog posts that things started to click into place.

I would be remiss if I didn’t also mention the fantastic Eric Florenzano who seem to get this much more intuitively then I do, and the amazing jChris who provides code to go along with his great ideas.

My desire to give back was peeked by Jan taking the time to answer a few desperate twitters I had. My first question was simple… why does some code say “map” while others use “emit”… simple answer; it was changed and “emit()” is now the proper syntax.

The second question occurred when I was querying a view with a map/reduce pair and couldn’t get the python library to return the key / value pairs to me as expected. As Jan states, if you pass “group=True” you can use results.key & results.value.

Ok, so that’s the simple Q&A recorded for posterity, but what about the neuron bending I promised. First let me say that if you want a tutorial on Neural Nets and AI programming you’ve come to the wrong place, and secondly this is my “answer” that’s evolved more then I care to admit.

So I’ll try not to take you through the brain process I went through but I’m sure we’ll both see how this could be made more “map/reduce”-y with future iterations. If you’ve got some of those insights I’d love to hear them because most of the examples I found already had that “AhhHA” moment made obvious.

One last disclaimer – I was looking to practice my jQuery and Django programming too so while this could definitely be made more “compact” it met my qualifications of building an “AJAX web application built with Django and utilizing CouchDB” application which was “simple”.

Now onward and upward! You’ll want to start a Django program to hold all this code, a process which is better covered in many other tutorials. The title for my project is “nn_clicks” and I creates a “clicks” application underneath this directory.

So after you’ve got your project made, let’s start with the “front-end” which in this modern age is nearly always the web browser. Of course we need a webpage, and you can find mine here.

It’s nothing fancy but you’ll see there are two HTML elements which will report coordinates on the page. One, “#loc” obviously tracks the mouse and the other, “#guess” holds the contents of our AJAX call, which won’t work now given my blog’s hosting provider.

The magic happens thanks to two jQuery calls. The first simply links the ‘#loc’ element to report our mouse locations;

$(document).ready(function() {
$().mousemove(function(e) {
$(‘#loc’).html(e.pageX +’, ‘+ e.pageY);
});

While the second is where the magic happens when a click occurs;

$().click(function(e) {
loc = $(‘#loc’).html(e.pageX +’, ‘+ e.pageY);
$(‘#loc’).animate({left: e.pageX, top: e.pageY}, 450);

$.post(“process_click”,
{ “X”: e.pageX, “Y”: e.pageY },
function (r, status) {
guess = $(‘#guess’);
guess.html(r.X + ‘, ‘ + r.Y);
guess.animate({left: r.X, top: r.Y}, 450);
},
“json”
);
});

This function first sets the location information for #loc again, just in case, and then proceeds to move that HTML element to where you clicked (a fun effect). For this to work you have to have the CSS “position” attribute set to “absolute”. Which is something I’ve done with inline CSS, which is ugly and poor practice for my style guidelines, but sufficient for this tutorial.

The call to the $.post(…) function handles the AJAX magic. It packs the click coordinates (which are really a measure of ‘X’ and ‘height’ rather then a strict X,Y interpretation) into a JSON structure and the part that reads; “function (r, status) { … }” is then called when the django call “def process_click(…)” completes (more on this later).

So copy the HTML file to the “templates” directory under your Django project (make it if this doesn’t exist) and edit your urls.py file to include these two lines;

(r’^process_click/?’, ‘nn_click.clicks.views.process_click’),
(r’^(.*)/?’, ‘nn_click.clicks.views.index’),

The first line will receive our jQuery POST call which has the mouse coordinates as the data and the second will send anything else to our main page. So now we can edit the applications view (vi clicks/view.py) and add the next phase of changes.

Here’s the necessary line to take care of the second url redirection;

def index(request, something):
return render_to_response(‘nn_click_template.html’, { })

This simply takes our template and returns it with room for future data (the empty “{ }” part) if I need it later. That was simple and if you start up your Django project you should be able to load the webpage and it will at least follow your mouse. Since I serve files from a different box then I use for development try; “python manage.py runserver 0.0.0.0:8000″.

Now let’s figure out how to process that POST data we get from our jQuery POST call.

Again in click/view.py add this code;

def process_click(request):
click_X = int(request.POST['X'])
click_Y = int(request.POST['Y'])
return HttpResponse( “{ “X”: %s, “Y”: %s}” % (click_X, click_Y)

Now the POST call comes in, we parse out the X & Y values and return that data to the page (asynchronously). What should happen is that shortly after the click you’ll see the second tuple move to the same location. You may decide to divide the data by 2 or swap the X, Y values for a little bit of fun.

That seems like a natural place to conclude part one and I know we didn’t get into the couchdb part but never fear I’ll have part two out shortly!

Update: I have this part running (minus the couchdb and neural net code since I can’t figure how how to get those working via fastCGI) but you can check out the basic idea here;

http://nn-click.thecapacity.org/

Posted in code, couchdb, python | 2 Comments

rsync is even smarter then I thought!

I love rsync and feel like I use it a lot but it took a friend of mine to teach me another lesson in the power of this tool! I think one of the reasons social networks have taken off is that they enable you to quickly poll a pool of active brain trust prior to hitting up google.

Even better this was a case of me expressing my frustration (via a facebook status message) and a friend proactively reacting to my discomfort!

My situation was this; I bought my wife one of the new macbooks… and how does that lead to rsync you might wonder? Well it’s complicate but it’s all about iTunes…

Our first experience with Apple and iTunes was when I bought a nano and a Nike+ system for her running. She has (soon that will be past tense) the only Windows system in the house (I run Linux) and she happily and blissfully (which means I wasn’t involved) loaded up all our music, which was a collection of years of random MP3′s.

Eventually, after I saw how neat the Nike+ system was, got my own nano and transmitter and since I didn’t know how iTunes would react to multiple devices I created a second user on her system and imported the same set of songs.

Now, give me a bit of credit… what I tried to do was point my iTunes to the same directory where her files were and hope that just the *.xml files would be different.

However, either I got this wrong or iTunes is silly (I’m leaning towards the latter) but it wasn’t the stellar setup I expected. Even worse, was that I had cleaned up all the files (Genre’s and so forth) and it didn’t reflect in her iTunes….

Now there we were, happy iPod users, and we eventually expanded our experience to include two new iPhones! When we activated them we used her iTunes (I wasn’t sure if our “Family Plan” would work with multiple iTunes). That’s when I learned iTunes is mostly ok with multiple users and multiple devices (except when it comes to podcasts).

So why rsync, and what does this have to do with her macbook? Well I decided to import “my” version of all the mp3′s since the tags and duplicates were mostly cleaned up, however since then she’d purchased songs and imported more music.

Thus, even after I’d imported a “clean” set of base files I still had to find a way to import her library, just minus the ones that had already been imported.

Thus my cry of pain…

The magic? I won’t bore you with mounting directories over the network, etc… but the magic line is this;

rsync -avhmP –ignore-existing –compare-dest=/home/itunes/iTunes Music/ /mnt/chaos/Documents and Settings/Administrator/My Documents/My Music/iTunes/iTunes Music /dump/remaining_iTunes/

Where “~iTunes” is the subset of “clean” files, “/mnt/chaos/….” is a cifs mount of her “dirty” files, and “remaining_iTunes” is where the difference b/t the two directories should go.

It’s not 100% completed yet but I think it’s working perfectly!

Thanks friends on the intertubes!

Posted in sysprog | Comments Off

Booklist – Entry 3.5: “Habits of the High-Tech Heart”

I’d like to take this brief interlude to make a painful admission to you. You may be asking how one could have a “.5″ book review… Well, I opened a book but actually didn’t read it!

In my youth, such a situation was cause for major turmoil, but thankfully as I’ve gotten older it’s become easier for me to admit something is a “waste” of my time and to move on without needing to turn that final page for true closure.

However, I’m still loath to perform such an action as I’m one who believes in “expanding your knowledge horizons” and I often feel aversion is a sign of successfully encountering such a stretching perspective.

Yet, in this case things were different. I won’t bore you with the positive comments on Habits of the High-Tech Heart: Living Virtuously in the Information Age since there’s plenty on Amazon to draw you to it. But I will say I’d managed to put this book out of my mind until I was cleaning out my Amazon list in preparation for the holidays.

What I did admit is that I checked this book out, read the introduction, and promptly gave up in disgust and if you’re at all intrigued by the reviews then I’d encourage you to try the intro at a local library before purchasing.

This is a book intended to remind us all to “cut free” on occasion and the review says the author “does not advocate the eradication” of technology.

However, what I found in the first pages was an obvious perspective that although technology is here to say and has many emotional benefits that the only rewarding path is to short circuit its use and circumvent it entirity.

It may be that the bulk of the book is more even keeled or I mistakenly judged too harshly, but I decided that I’m well enough versed on the perils of our technological landscape.

I do practice cutting free and try to remember to experience events and not simply record them. However, I think it speaks of false hope and misconstrued expectation to believe you can only find goodness outside of, or without. technology.

I’d love to hear about your experience with this book if anyone manages to give it a go but it certainly won’t be one I’d recommend putting on your holiday wish list.

Posted in books | Comments Off

Booklist – Entry 3: The Drunkard’s Walk

This is the second book in a row that I’ve really enjoyed so it might be easy to say I was on a roll picking winners. Yet, to do so would negate all I’d learned from “The Drunkard’s Walk” which is a fantastic journey into “how randomness rules our lives”!

First, let me start with the ending which directly addresses the question that always springs to mind when people explain how randomness underlies most everything, even supposed patterns or successes. I’ll just mention that Leonard Mlondinow concludes his book by confronting this concept in a heartfelt manner and I can only encourage you all not to lose hope by the rest of my post.

The term “Brownian Motion” is a more polite term for the drunkard’s walk, but I’m sure many of us are familiar with the concept that we often get to where we were going without always intentionally steering ourselves there directly.

Before I started my book list entries I read “The Black Swan” based on a friend’s recommendation. It’s a good book too but I felt like it left me with more questions (although with more respect and caution, and my second book gave me a sense of how our underlying psychology can “randomly” (or not so randomly) influence our lives.

This book really seemed to pull a lot of those same concepts into both a historical and tangible set of examples.

The author, Leonard Mlodinow, brings to our attention the risks of confirmation bias” and how often it influences our expectation of whether something is or isn’t random.

About the only point where I “disagreed” (i.e. where I wanted to have a more in depth conversation) was when he mentions that firing high level managers shows a 50% rate of improvement.

His point is that this shows that managers don’t always have any influence whatsoever but I wonder if it simply means that only roughly 50% of all managers are good.

In short I think Leonard falls prey to something he often guards against if you use statistics to generalize (as he did here) then you miss out that often results can go “counter” to the trend for the individual, i.e. instead of “breaking even” you could see a 100% improvement in you rcompoany (probably at the expense of another which ends up @ -100%).

However, given that I’m not a mathematician I’m sure I’m falling prey to the influences that we all suffer from, understanding how randomness predominates our existence. Clearly I could use some more reading on this subject but this book is a great start in that direction!

Posted in books | Comments Off

Booklist – Entry 2

As promised in my earlier post I wanted to be more active in continuing my reviews, and this is a book worth the accolades.

In “Predictably Irrational” the author, Dan Ariely, gives us a chance face some of the most astounding incongruities of the human persona.

I couldn’t do all the examples justice but let’s take one of the “classic” observations. Assume you’re given a description of a woman who was politically and environmentally active in college. You even learn that she was arrested once for chaining herself to a tree! Then you’re asked which of these futures she’s most likely to have;

(a) Working in the finance industry as a bank teller

(b) Single parent working as a counselor for battered women

(c) Doing social work

Now I’m not the scientist Dan is but assuming I’ve not completely missed the tone, many, even most, of us might answer option (b) as being the most probable path of our fictious female.

However, if we step back from our human instincts we realize that (b) is actually a more “specific subset” (my words not the authors) of option (c)!! So because of it’s broader categorization picking (c) stands a better chance of being correct but because (b) appeals to our “sense of story” we tend to gravitate towards it being most probable!

Short of recreating the book in a blog post there’s no way I could do these examples and insights justice but if you enjoy learning about how we’re “mislead” (a polite term) as consumers, or why we don’t always make “logical” decisions (though they are as the title suggests “irrational in a predictable manner”) then this is the book for you!

It’s a well written book which has digestible chapters and a story like progression that make it enticingly easy and insightful to read!

Posted in books | 4 Comments

< /BookReview 1 > *bug* You can’t make up fake tags in WP

You can probably tell from the picture that my wife and I have done a lot of traveling. I actually finished Superclass a few weeks ago. Yet, I’m embarrassed to admit it’s been over a month since I proclaimed my intent to review the books I’ve been reading.

However, I’m determined not to interleave reviews so let me comment on this one so I can get on with the others! In fact, I’m quite certain I can review this in far less time then it took me to complete.

I really respect the tone with which David Rothkopf approached his topic. Writing rationally about a global elite which could move mountains, especially in this day and age, is a tough line to walk.

I also think David gives well weighted credibility to debunking the “conspiracy theories” as well as highlighting the dangers that such a connected class brings whether intentionally or accidentally.

Despite his even handed approach I wish I could have found the technique as enjoyable. I’m not someone who enjoys “the name game” or following cliques and lists of “who’s who” and perhaps that made this book extra tough for me.

The first and last few chapters are pretty enjoyable but the meat of the book is was very difficult for me to make it through. I’d recommend the book to anyone who’s interested in this topic but it’s a perfect example of the benefits of your local library!

Posted in books | 1 Comment

Apple’s lone Genius

Cool visualizations aside, the most interesting announcement of iTunes 8 (and the quick subsequent bugfix) was the new “Genius” feature.

Steve Jobs has been criticized for being out of touch before, when the iPhone came out with out any sort of social networking. I vouch that the much vaunted Apple “experience” breaks down at the usage fringes, i.e. if you’re not using it to listen to your extensive music collection it’s pretty rough around the edges.

What Genius does is mix and match music based on beats and other metrics and builds playlists. I listen to podcasts and audiobooks so I can’t speak from real experience but word is that it’s neat and amazingly successful.

However, even after all the successes of “social networking”, Genius really seems to highlight Apple’s interesting lack of desire to build collaboration in their products.

I find it shocking that there’s absolutely no “social networking” features like Last.fm. It seems Apple thinks math can be a better friend to you then your own friends (like that time when they rickrolled you).

As often as I send links to friends with delicious, or videos on YouTube, you’d think a day had come with you could nudge your friends to try out this song… well of course Zune tried that but failed.

However, one believes Apple could “do it right” and the fact that they haven’t even tried seems suspicious. It seems that until they do, whether it’s from “Top 40″ count downs or inhuman algorithms, we’ll forever be forced to take “the man’s” word for what’s cool.

Posted in Apple, social | Comments Off

What do I win… ?

I have a confession to make which probably won’t surprise many of you, least of all my wife. I hope it’s a minor flaw that doesn’t make me too intolerable but the fact is: I like to be right.

I don’t believe I’m the “In your face – *boo yah* Where’s my $20?” kinda right. I prefer to consider it as a paternal smugness, quiet calmness with a wry, knowing “Well, I’m sure you had to figure that out on your own but I just wish I could have made things easier for you” kind of smile.

So I don’t normally go out of my way to toot my own horn but I wanted to take exception because I think it will illustrate a lot about me… and trust me I’m wrong a fair amount of the time too.

There are great debates about “Generalists vs. Specialists” and “Consumers vs. Producers” and I’m sure a ton of other “Mavens, Connectors, etc…” generalizations from Freakenomics.

Honestly, I’m not sure which of any of those labels I truly am. On a Meyers Briggs test I’m a bit of everything and I sort of think of myself as a guy who could step into many situations and make do, which probably means a “Consuming Generalist Connector” if one had to pick.

To me it’s reminiscent of that TV show “The Pretender”, minus the genious IQ. I make no illusions that I’d be an adequate Dr. or working at the LHC, but I’ve done marketing, management, business, technology and lots of variations in between.

Most of my friends know that I parse a lot of data, it “feeds my need” and making the connections and tracking the patterns is something I find fulfilling. Recently, someone told me I have a great mind for strategy and I glossed over the compliment but in this most recent context it was interesting to reevaluate that statement.

So what was it? What did I do? Simply put, I did what I enjoy doing; I read and observed and let intuition guide.

I won’t bore you with a soliloquy about the merits of on demand fabrication and the future, but if you’ve followed any of it you’ve heard about Ponoko, BigBlueSaw and Shapeways.

Recently Ponoko announced an upcoming feature release, something to make digital design and fabrication easier and more approachable then invited people to guess. So I guessed, not having “the answer” in mind till I started writing my comment but when I wrote I had that sense of knowing I was right.

It wasn’t like competing for a Nobel Prize, there were probably only about 75 other posters but the only other person who mentioned something similar was the one immediately following mine. And now it’s real and people are excited about it.

Me, I think it’s a great concept and I’m excited too, but I must admit; I’m also excited about being right.

So Ponoko, where’s my prize?

Posted in career, inspiration, technology | Comments Off

Quick couchdb reminders

Using your browser to access the admin interface;

http://192.168.1.99:5984/_utils/

Looking at a view for a specific database with your browser. In this case the database is “stock_values” and the view is “all_stocks” (yes you need the “_view” and the “all” part of it);

http://192.168.1.99:5984/stock_values/_view/all_stocks/all

I was really getting tripped up by figuring out the corresponding python version which would be;

for ro in db.view(‘all_stocks/all’): print ro.id

Posted in code, couchdb, python | 1 Comment