ls -a

Inkage?

Tue, 30 Mar 2010 00:57:04 -0400

Contemplating a tattoo. Something witty/geeky, but not sure what to get yet. I’ll be adding to this list but so far here are my choices in no particular order. Reply with any helpful suggestions :)

1. Hello World

2. Mushroom or turtle from super mario bros.

Stuff I want

Fri, 05 Feb 2010 17:42:00 -0500

Fireball Island (Purchasing on Ebay as I’m posting this)

Any of these madballs

New hotness

Wed, 03 Feb 2010 12:28:00 -0500

Hey guys,

Just wanted to inform you of a new project I’ve been working on with @Davidedicillo and @orical.

As you may or may not know, location based services are the hotness right now. Foursquare is one of them, and it’s gaining traction reminiscent of Twitter circa 2007. The idea is that you go around and check-in to different venues with an application that resides on your phone. As you do this, your friends can see what you are up to and choose to join you. The one problem is that check-ins are text based and you can’t drop a photo or video at that location. What if you want to show everyone this great beer you’re drinking as opposed to just describing it.

We aim to solve that by providing an iPhone application that allows people to check-in with a photo or video, in addition the app will show users all photos and videos dropped at a particular venue. The app is called SquarePik it’s built on top of our API and will be available shortly in the iTunes store for just $1.99

Let me know what you think. I’ll post some redemption codes here as soon as it drops in the store.

Menomena - Wet & Rusting

Thu, 28 Jan 2010 15:43:00 -0500

Found this tune courtesy of Marie Lucas.

Sort of feel like dipping myself in and out of water when I listen to this.

Best cheesy inspirational songs

Fri, 08 Jan 2010 17:25:20 -0500

Decided to bring in the weekend with some kick ass motivational tunes. I still can’t believe Ralph Macchio is 50!

Had this song in my head all day.

Wed, 06 Jan 2010 17:18:00 -0500

By far amongst the best lyrics in history.

Cassandra Messaging Tutorial - Part 1

Mon, 04 Jan 2010 23:27:00 -0500

I am currently working on developing the basic architectural components for a project me and my buddies are putting together. Since forming relationships via communication is going to be our core business, the internal messaging system needs to be built on something solid that can scale out horizontally without the need for replication. Enough about that, the goal is to show how how to build out a simple internal messaging system using Cassandra.

Before anyone goes on a rant about the fact that I am thinking scale here, I want to make one thing clear. Yes, it is way to early to think about ways to scale a service before even testing out the waters. I am fully aware of this fact, and I am in no way endorsing you to think that your service will eventually scale before even putting a minimum viable product out there. In fact, research has shown that it probably will never scale beyond a couple hundred thousand http requests a day and about the same amount of read/write storage transactions.

This is more of a learning experience for me and also an excuse to start getting in to some more in depth hacking on distributed data storage systems using Cassandra as a means to achieve this.

Loose Pre-requisite

You are using Rails for your front facing presentation layer. (Or are familiar with it)

Strict Pre-requisite

You have read this, and you know how to fire up an instance of Cassandra server running on at least 1 node. The aforementioned post is by far the best quick start guide I’ve found on Cassandra. Props to Evan Weaverfor writing it.

Mindset

Remember, unlike RDBMS’s you have to think carefully of modeling the data to suit your application. You can’t just model relationships and add indexes to support queries that will be needed as you progress. Therefore de-normalization is absolutely necessary. Just remember any college level DB course you had and do the exact opposite when it comes to data normalization.

Messaging System Supported Use Cases

1. A user should be able to compose a message and send it to another user.

2. A user should be able to view incoming messages via an inbox, ordered by most recent.

3. A user should be able to view outgoing messages, ordered by most recent first.

4. A user should be able to view a detailed message thread including any of it’s replies.

5. A user should be able to reply to a message thread.

* For the time being, a user will not be able to search through messages. Search will be implemented later, probably with lucene. Not sure on this yet, but will write about it when I decide to implement.

* This also doesn’t cover attaching objects such as virtual gifts and images to messages. I Will be covering this in a later post, but it’s pretty straight forward.

Step 1 - Data Model

I decided to stick with a simple construct and use a single ColumnFamily. There are many ways to do this, including using a multi column family to structure the data. But I chose to keep it as simple as possible to store and display messages.

I set up 1 main Keyspace for the project and 2 ColumnFamily’s for the messaging system. The Columnfamilys are ordered with the TimeUUIDType comparator since we want them ordered by time.

Contents ofstorage-conf.xml:

Messages - This will hold user ids as keys and associate a unique sortable column name with a JSON hash containing the message details.

MessageReplies - Messages ids as keys with an associative unique sortable column name containing a JSON hash that will hold reply details.

This modeling will easily allow us to list Messages for a given user as well as details and replies to a given message, if any are present.

Step 2 - Usage and examples

To illustrate usage, I’ll fire up a ruby interactive prompt, and populate/retrieve data. (just type ‘irb’ in your terminal to get a shell going)

require 'rubygems'

require 'cassandra'

include Cassandra:Constants

require 'rubygems'

Instantiate a Cassandra client object:

messages = Cassandra.new('MainProject')

A few insertions

Let’s assume Jennifer, user id 1200 is sending Emmanuel, user id 1120 a message. What we do, is insert the message once postfixing ‘_in’ to Emmanuel’s user id, and then once more postfixing ‘_out’ to Jennifer’s user id. These two will be our keys. (Remember that a user can view both an inbox and outbox of messages and there is no notion of easily querying over values, therefore redundancy in this case is good). Notice that we are doing two successive insertions so some level of transaction control is needed when this is put in to production code.

uuid = UUID.new

messages.insert(:Messages, '1120_in', {uuid=>'{"username":"Jennifer", "age":"24", "location":"Aventura Florida", "subject":"Hey there","body":"Cool profile. Drinks sometime?", "read":"0"}'})

messages.insert(:Messages, '1200_out', {uuid=>'{"username":"Emmanuel", "age":"26", "location":"Miami Florida", "subject":"Hey there", "body":"Cool profile. Drinks sometime?", "read":"0"}'})

Let’s insert another message to the same user, but this time from Natalie:

uuid = UUID.new

messages.insert(:Messages, '1120_in', {uuid=>'{"username":"Natalie", "age":"25", "location":"San Francisco CA", "subject":"Do you like Ninjas?", "body":"It might be a dumb question. Just wanted to know if you did :-)","read":"0"}'})

messages.insert(:Messages, '1320_out', {uuid=>'{"username":"Emmanuel", "age":"26", "location":"Miami FL", "subject":"Do you like Ninjas?", "body":"It might be a dumb question. Just wanted to know if you did :-)", "read":"0"}'})

So far we took care of use case #1. Next, let’s assume Emmanuel replied to Natalie’s message:

messages.insert(:MessageReplies, uuid, {UUID.new=>'{"username":"Emmanuel", "age":"26", "location":"Miami FL", "body":"I love Ninjas! OK we can officially go grab a drink now!"}'})

This takes care of use case #5. Next we’ll move on to cover the last 3 use cases via some basic retrieval operations.

And now for retrieval

Now I can easily retrieve all messages for a given user based on user id’s and a postfix of ‘_in’ or ‘out’. After that I can iterate over them and display messages however I want. This takes care of use cases #2, and #3.

For Emmanuel’s Inbox:

collection = messages.get(:Messages, '1120_in', :reversed=>'true').to_a

For Emmanuel’s Outbox:

collection = messages.get(:Messages, '1120_out', :reversed=>'true').to_a

Onto covering use case #4, let’s say you listed out a user’s messages on a page and upon clicking on an entry you want to display message details and replies, a sort of permalink for a message.

All we need now is the messages UUID, we can get this by calling to_guid, which will generate a readable representation of a Cassandra UUID.

For simplicity, let’s access Emmanuel’s first result, and get it’s message guid:

guid = collection[0].first.to_guid

Now you can use the guid to retrieve message details and any replies associated with it. We can make use of the only supported range query Cassandra has (Return a range of ids and limit the result set)

msg_details = messages.get(:Messages, '1120_in', :start=>UUID.new(guid), :count=>'1', :reversed=>'true').to_a

Last step is to retrieve all replies for this message, and to illustrate limiting the result set lets return 20 latest replies.

guid = msg_details.to_a.first[0].to_guid

replies = messages.get(:MessageReplies, guid, :count =>'20', :reversed=>'true').to_a

That’s pretty much it. I’ll respond to the comment thread if anything needs clarification.

Next post will be about adding attachments to the message structure, which is going to be fairly easy, given that the meat of the module is done.

Really re-thinking certain key things in my life for 2010....

Mon, 04 Jan 2010 11:39:00 -0500

Really re-thinking certain key things in my life for 2010. Embracing change.

Bear - The Antlers

This song is pure bliss, with an overtone of despair. Wildcat -...

Thu, 31 Dec 2009 16:25:52 -0500

This song is pure bliss, with an overtone of despair.

Wildcat - Ratatat

Listening to some new additions to my collection courtesy of...

Sat, 26 Dec 2009 14:08:00 -0500

Listening to some new additions to my collection courtesy of Marie. My favorite holiday gift thus far.

Spam

Fri, 18 Dec 2009 16:18:00 -0500

Me: I just got an email: "Your zoosk matches are waiting", we need to start spamming people! This stuff works.

Peter: Spam people... Check. You can't spell success without spam.

Me: gem install mail_spammer

Peter: Error : Dependency Check... "It seems you have morals, this gem requires you to run 'gem remove morals' before you continue."

Me: Weird.. I don't get that error.

Scaling on up [part 1]

Wed, 04 Nov 2009 17:05:00 -0500

For a recent ongoing project, I realized that I had to begin offloading a sub-system from mySQL to a key/value store for efficient retrieval. We all know that mySQL heavily depends on power to read as opposed to write. Therefore, when dealing with joins, and sometimes IN’s on non trivial queries, it can become fairly expensive. Not to mention my budget is finite, so I can’t (and wouldn’t, even if I could) just throw metal at the DB.

The sub-system in question is a user’s activity stream. A user has friend’s those friends perform activities such as post photos, comments, be-friend other people etc. Naturally to display all this data from mySQL, lots of joins are required from various tables Users,Friends,Photos,Comments. An alternate solution is to employ a mySQL strategy which keeps track of redundant fields in a separate table that is populated asynchronously. With this method, all data can be pulled in a simple SELECT, again this is nothing more than just a patch, and not really a viable long term solution for scaling.

The real way to alleviate this, is to replace the way activities are displayed, and persisted with an alternate key/value store that is quick to serve data from even when sometimes retrieving non-cached content. In addition, it has to be an augmented key/value store, so that it somewhat facilitates basic relationships.

So now which one do I choose? There are a couple of factors that lead to my decision:

1. Something with very low latency on reads.

2. Something that plays nice with THRIFT or HTTP.

3. A system that is truly distributed and fault tolerant.

4. Eventually consistent system, since availability is more important than inconsistent reads.

5. Decent developer community contributing to the project.

After looking at a couple of options, I narrowed it down to two; Cassandra, or Riak. Both great distributed systems, one system battle tested (Cassandra), the other not so much, but both very much still experimental. I am familiar with Cassandra from a previous endeavor, so I know what it can do. It is also proven to handle well in production (Facebook,Twitter,Digg), so it’s a solid contender, but I’m not going to go with it until I experiment with Riak a bit more. Right off the bat, I was impressed by some of RIAK’s core features, such as virtual node replication and linking between documents, just to mention the attractive ones. The only drawback so far is the lack of documentation and community, since it is so new.

I will have to make a decision soon, and when I do, a post will follow with some more technical details about the move and which system I eventually chose. Until then I’ll be hacking away at Riak to see if it’s a viable option for putting something like this in production, if not then good ol’ Cassandra it is :-)

A new addition to the family. My 1972 Raleigh Super Course with...

Wed, 28 Oct 2009 10:48:00 -0400

View #1

View #2

Rear Tire View

Front Tire View

Brooks Saddle

A new addition to the family. My 1972 Raleigh Super Course with a fresh coat of old English racing green. Wondering what I should name it..

The Bucket List

Mon, 12 Oct 2009 23:42:00 -0400

Here is a list of the 10 things I’d like to do in my lifetime, in no particular order.

1. Write a book

I have no idea what I would write about, but it would definitely be gratifying.

2. Write a horror movie screen play

As some of you may know, I have a passion for all things gore and horror, I have tons of sick ideas that could really be put to good use.

3. Be involved in a music project

Wether directly or indirectly I want to be involved in a music based project. I have no musical talent whatsoever but that doesn’t stop modern day artists does it!?

4. Sky dive

Because I wanna come as close to flying without attempting a base jump.

5. Travel the world for at least a year

Live and experience everything through different perspectives is the goal here.

6. Live in San Francisco, NYC, or Paris

These are my 3 favorite cities, all for different reasons, I don’t know if one stands above the rest as of now.

7. Open a restaurant or small bookstore

The minute I have a sizable amount of disposable income this is what it would go in to. I have always wanted to do this at some point in my life.

8. Get invited to TED

By far some of the brightest minds in science, business, social issues, technology and more. This is one conference I would love to attend just for the sake of absorbing.

9. Finish my PHD

I am currently on leave from my PHD, I left because I wasn’t ready to dedicate myself to a life of research and academia. I do however want to revisit my studies because I know I have so much more to contribute and learn.

10. Be more of a humanitarian

I definitely lack in this department, and would like to make up for it down the line.

CakePHP Error Handling

Mon, 28 Sep 2009 23:14:00 -0400

So a couple of people asked me about the correct way of handling errors in CakePHP, and defining custom error pages for websites, so I decided to not explain this any more and just hand out a url to this post.

Using standard error messages

For common errors such as 404, Controller not found etc, you could easily use CakePHP’s built in cakeError function. So if you wanted to define a 404 error you would simply call the following from your controller or component:

$this->cakeError('error404');

Ok, this is all good, but what if we want more control over error handling, what if we want to define our own error handlers.

Creating custom error messages

1. Extend the default error handler by creating a file: app/app_error.php with the following definition:

class AppError extends ErrorHandler { }?>

2. Now that you have this class all you have to do is define custom error handling functions. Include as many custom errors in this class. So lets take a situation where a user is not found in the database when people try to access domain.com/username from your site root.

function userNotFound($params) { $this->controller->set('file', $params['username']); $this->_outputMessage('user_not_found'); }

$this->_outputMessage() will just display the view found in views/errors/.ctp

3. Next create the views and override the default layout.

app/views/user_not_found.ctp

`User not found`


Sorry, but we couldn't find  
in the system.

If you want to customize the layout for your errors, simply create app/views/layouts/default.ctp and include any custom css you want. Don’t forget to include the standard $content_for_layout and $title_for_layout variables in your layout.

4. Call your custom functions from your controllers or components:

$this->cakeError('userNotFound', array('username'=>$username));

That’s pretty much it, you can now handle errors easily and elegantly :-)

This always reminds me of sitting in a car, outside of certain a bowling alley in 2006.

Fri, 25 Sep 2009 00:38:30 -0400

One rough pigeon.

Thu, 24 Sep 2009 17:13:23 -0400

One rough pigeon.

Some Cardboard Lovin

Mon, 21 Sep 2009 12:35:00 -0400

I came across a great find today. Cardboard love, it’s by a buddy of mine Joey Hall who also happens to be the king of all fashion, just in case you need any glamor advice.

So hit up Cardboard Love and let me know what you think.

Great fashion resource IMO

Fri, 18 Sep 2009 12:34:00 -0400

Great fashion resource IMO:

Abnormality at it’s finest. Via Pikchur user...

Thu, 17 Sep 2009 15:29:00 -0400

Abnormality at it’s finest. Via Pikchur user @dashingclaire