oracle aide

November 3, 2013

It is not Big Data, it is Slow Data ©

Filed under: Uncategorized — oracleaide @ 10:27 pm

Just sayin’…

For an average human it is hard to fathom the volume of data he deals with.  

The notion of “a lot of data” changes with Moore’s law and highly subjective: from a stack of punch cards to a rack of hard drives.

A gigabyte used to mean “a lot of data”.

Not anymore.

What  an average human could fathom is his personal perception of how fast it takes to process data. 

Thus, while working with Hadoop based technologies I couldn’t help noticing — how long it takes to process small samples — comparing to relational databases.

Which is the small (but annoying) price to pay for the overwhelming speed of processing “a lot of data”.

That is why it is critical to run pig in local mode when going through tutorials.

pig -x local

Advertisements

Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: