Triangle Hadoop Users Group

Slides from May 22nd talk by David Arthur

Thu, 24 May 2012 12:41:00 -0400

Thanks to David Arthur of Lucid Imagination for his presentation on Apache ZooKeeper!

Intro to Apache Zookeeper

Mon, 21 May 2012 08:41:00 -0400

ZooKeeper - An Introduction and Practical Use Cases”

Speaker: David Arthur, Lucid Imagination

Date: May 22nd, 6:30PM

Location: Bronto Software, Durham, NC

RSVP: http://trihug-05-2012.eventbrite.com/

Abstract: ZooKeeper is a distributed coordination service with a strong emphasis on update consistency. It can be used for simple things like configuration management and distributed id assignment, as well as more complex things like distributed locking and service discovery. The API provided by ZooKeeper is rather low-level and requires a bit of boilerplate code, so we will also look at some higher level frameworks.

David Arthur is a software engineer at Lucid Imagination working on the new “Big Data” team. He has been working for the last two years building distributed systems where Hadoop has been a central component. Prior to working in the “big data” space, he focused mainly on data side of applications: schema optimization, data warehousing, etc. He attended Florida State University where he received a B.S. in Physics and completed two years of graduate studies in Scientific Computing.

Next Meeting: IBM Watson on April 12, 2012 @ Bronto Software

Wed, 04 Apr 2012 15:38:24 -0400

Title: IBM Watson: Big Data Text Analytics

Location: Bronto Software in Durham, NC

Speaker: John Gerken

RSVP: http://trihug-04-2012.eventbrite.com/

Abstract: What is IBM Watson? It managed to defeat two previously undefeated human opponents in a game show and it has shown how super computers are becoming more and more able to understand and answer questions in ways previously reserved for the domain of human thought. But what most don’t realize is how the synergy between Big Data and text analytics was integral to enabling IBM Watson’s capabilities. Curious as to how Watson leveraged Big Data and text analytics technologies? Wondering what the future may hold for applying them? If so, attend TriHUG on April 12 to find out.

Speaker Bio: John Gerken is a Senior Software Architect in IBM’s Emerging Technologies jStart Team, where he is responsible for recognizing, promoting and developing prototypes of software technologies and trends that could positively impact IBM’s customers. John is an IBM Watson Team Leader working to enable Watson to be used by customers. He is also a recognized thought leader in the area of Situational Applications and mashup ecosystems and is a principle evangelist for these technologies. John is a member of the North Carolina Technical Experts Council (NC TEC), which is an IBM Academy affiliated technical advisory and vitality organization serving the RTP, NC area. He also holds a Bachelors of Science in Jazz Performance and plays at every opportunity.

Sponsors: Bronto Software, Lucid Imagination

Slides from March 20 talk by Jameson Lopp

Fri, 23 Mar 2012 11:50:23 -0400

Thanks to Jameson Lopp of Bronto Software for his presentation on Pratical Pig. Here are the slides:

Pratical Pig

View more presentations from trihug.

Slides from Feb 16. talk by Adam Gugliciello of Datameer

Wed, 22 Feb 2012 19:55:28 -0500

Thanks to Adam Gugliciello of Datameer for the talk at TriHUG last week! Here are the slides:

Financial services trihug

View more presentations from trihug.

Next Meeting: Feb. 16 @ Bronto Software

Wed, 01 Feb 2012 08:37:00 -0500

Title: Financial Data Analytics with Hadoop

Sponsored By:

RSVP here

Abstract:

Hadoop based applications are becoming critical in the financial services arena for the analysis and correlation of large volumes of structured and unstructured data. In addition, the Dodd-Frank Act signifies the largest US financial regulatory change in several decades and requires much greater transparency on financial data. In this session, we will answer common questions and demonstrate use cases in how Hadoop and Datameer help with asset management and risk management, fraud detection and data security.

Leave this session knowing about:

Financial data and Hadoop. What data lends itself to Hadoop? What doesn’t?
Benchmarks from real-world uses of Hadoop in finance
How to effectively migrate, manage, and analyze financial data using Hadoop

Bio: Adam Gugliciello, a 15-year veteran in Software Engineering and Systems Architecture specializes in highly available, parallel systems. Most recently he has been developing grid computing solutions to enable deep analyses and intelligence gathering on huge software systems for technical debt and functional mapping. Adam is a Solution Engineer at Datameer and helps bring Financial and Telco applications expertise to the utilization of the Datameer business intelligence suite.

Slides from Intro to HBase presentation January 2012

Wed, 18 Jan 2012 14:04:00 -0500

Thanks to Chris Shain from Tresata for coming to Durham last night to talk about HBase.

TriHUG January 2012 Talk by Chris Shain

Next Meeting: January 17, 2012 @ Bronto Software

Sun, 08 Jan 2012 16:54:00 -0500

Title: Intro to Apache HBase by Chris Shain of Tresata

Location: Bronto Software in Durham, NC

RSVP

Abstract: Chris will provide an introduction to Apache HBase, aiming to discuss:

What is HBase? (High level overview)
Details of the HBase architecture
How do clients interact with HBase?
Some general HBase patterns and anti-patterns
What are the use cases for HBase vs. Relational DB?

Bio: Chris Shain is the software development lead at Tresata, a provider of Big Data solutions for the financial industry in Charlotte NC. His background includes 7+ years of software development experience in the financial services industry, with a focus on customer-facing data management applications and data warehousing. Lately he works with Hadoop and HBase on data volumes in the multi-terabyte range, and tinkers with geographic information systems. He lives in Charlotte NC, and can be reached at chris@tresata.com or twitter @ChrisShain.

Slides from Alan Gates Presentation on Nov. 15, 2011

Fri, 18 Nov 2011 15:20:38 -0500

Thanks to Alan Gates of Hortonworks for the two excellent presentations on Apache Pig and Apache HCatalog. Links to the slides for the two talks are included below and are also available on Slideshare.

TriHUG November Pig Talk by Alan Gates

View more presentations from trihug.

TriHUG November HCatalog Talk by Alan Gates

View more presentations from trihug.

Slides from Oct. 11 TriHUG meeting featuring Josh Patterson of Cloudera

Wed, 12 Oct 2011 15:53:37 -0400

OSCON Data 2011 - Lumberyard View more presentations from Josh Patterson

Next Meeting: November 15, 2011 @ Bronto Software

Wed, 12 Oct 2011 08:18:00 -0400

Our next meeting will be November 15 at Bronto Software. The speaker will be Alan Gates, the author of Programming Pig and a member of the Hortonworks team. RSVP here.

————-

Title: New Features in Pig 0.9 and Introducing HCatalog

Abstract: Pig 0.9 added several features to make Pig a more powerful data processing platform, including macros, include statements, and the ability to embed Pig in Python for control flow. We’ll cover these, talk about some new features that have been added since 0.9, and what’s next on Pig’s roadmap.

HCatalog is a table management and storage management layer for Hadoop that enables users with different data processing tools – Pig, MapReduce, Hive, Streaming – to more easily read and write data on the grid. HCatalog’s table abstraction presents users with a relational view of data in the Hadoop distributed file system (HDFS) and ensures that users need not worry about where or in what format their data is stored – RCFile format, text files, sequence files. This talk will include an overview of HCatalog’s features and a discussion of its current roadmap.

Bio: Alan is a co-founder of Hortonworks as well as an original member of the engineering team that took Pig from a Yahoo! Labs research project to a successful Apache open source project. Alan also designed HCatalog and guided its adoption as an Apache Incubator project. Alan has a BS in Mathematics from Oregon State University and a MA in Theology from Fuller Theological Seminary. He is also the author of Programming Pig, a forthcoming book from O’Reilly Press. Follow Alan on Twitter: @alanfgates.

TriHUG Next Meeting featuring Josh Patterson of Cloudera set for Oct. 11

Wed, 14 Sep 2011 08:26:45 -0400

The next Triangle Hadoop User Group meeting will be October 11th at Bronto Software and will be featuring Josh Patterson of Cloudera. RSVP here.

Title: Lumberyard: Time series Indexing at Scale

Abstract:

As time series data explodes in volume in the genomic, sensor, and

financial realms [1] companies are looking for more effective ways to

store and query this data. To handle this explosion in scale systems

are looking to the Hadoop, HBase, and NoSQL domain for components to

build their systems on. In this talk we introduce Lumberyard [3], a

system which can potentially (1) store Terabytes of time series data

and allow for this data to be interactively queried at low latencies

to provide real time access. Lumberyard stores iSAX [4] indexes in

HBase’s Multi-dimensional sorted map storage system which give

Lumberyard the reliability of HDFS yet the low latencies of HBase. Our

approach leverages a multidimensional indexing structure which is

stored in HBase’s highly available distributed multi-dimensional

sorted map. We present the design of Lumberyard’s implementation and

illustrate the differences between an in-memory iSAX index compared

with a persisted HBase-backed iSAX index.

Sponsored by Cloudera and Bronto Software.

More info at www.trihug.org.

Bio:

Master’s Thesis: self-organizing mesh networks Published in IAAI-09:

TinyTermite: A Secure Routing Algorithm

Conceived, built, and led Hadoop integration for the openPDC project

at TVA (Smartgrid stuff). Led small team which designed classification

techniques for timeseries and Map Reduce. Open source work at

http://openpdc.codeplex.com

Now: Sr. Solutions Architect at Cloudera

Slides from Ted Dunning's Sept. 2011 talk

Wed, 14 Sep 2011 07:15:48 -0400

Thanks to everyone for attending last night’s talk! Ted’s slides are available for download below.

MapR, Implications for Integration View more presentations from trihug

Starfish Talk Slides from April 2011

Fri, 09 Sep 2011 08:42:13 -0400

Under the better late than never category, here are the slides from the April 2011 TriHUG meeting on Starfish.

Starfish: A Self-tuning System for Big Data Analytics View more presentations from gsingers.

Next Meeting: Sept. 13 @ Bronto Software

Sun, 28 Aug 2011 15:46:23 -0400

We trust everyone has had a good summer and is equally excited to get back into learning more about Apache Hadoop and scaling. Our next meeting will be Sept. 13 at Bronto Software. Food and drinks start at 6:30 and the talks start at 7.

We are pleased to announce that our speaker will be Ted Dunning from MapR Technologies. See below for more details. Please RSVP here.

Title: MapR, Architecture and Implications

Abstract:

The talk will be a description of how MapR’s architectural advances allow significant improvements in speed, reliability and scalability over stock Hadoop. This will include a dive into the MapR file system and a discussion of how the map-reduce layer has been changed and the impact on other Hadoop eco-system components. This will include actual test results.

In the second section of my talk, I will describe how this new architecture has surprising consequences. In particular, I will show how tasks like machine learning, data visualization and search indexing can all work better on the MapR platform.

Ted’s Bio:

Ted has held Chief Scientist positions at Veoh Networks, ID Analytics and at MusicMatch, (now Yahoo Music). Ted is responsible for building the most advanced identity theft detection system on the planet, as well as one of the largest peer-assisted video distribution systems and ground-breaking music and video recommendations systems. Ted has 15 issued and 15 pending patents and contributes to several Apache open source projects including Hadoop, Zookeeper and Hbase. He is also a committer for Apache Mahout. Ted earned a BS degree in electrical engineering from the University of Colorado; a MS degree in computer science from New Mexico State University; and a Ph.D. in computing science from Sheffield University in the United Kingdom. Ted also bought the drinks at one of the very first Hadoop User Group meetings.

RTP Scaling Hackathon (Planning Stages)

Wed, 11 May 2011 18:29:16 -0400

Some TriHUG members are in the early stage of putting together an all day hackathon on all things scaling (Hadoop, Cassandra, Hive, Pig, Mahout, etc.) and wanted to get some info out to the community as well as a call for volunteers and sponsors.

The basic gist of the day is that we get together and spend the day hacking and learning about writing scalable, fault tolerant systems. All ranges of experience are welcome and we fully expect that one of the groups that forms will be a “tutorial” group, while other groups will be doing more advanced things. The key is to get lots of interaction and cross-fertilization of ideas.

Our tentative plan is that we will make available:

1. Compute Cluster time (likely Amazon EC2) along with ready to use instances w/ appropriate things already installed. (More later)
2. Some public data sets, but feel free to bring your own publicly available on Amazon S3
3. Food, drinks, etc. including pizza/beer at the end
4. Network connectivity
5. Space to work in
6. (TBD) Machine to submit jobs using a fair scheduler

You need to bring your laptop and an open mind. Also having your favorite tools on your machine would be good. A github account or something similar would also be useful.

Our likely date for this is June 18th with a backup date of June 25 (pending space availability) from 9 AM - 6 (?) PM. Attendance will require RSVP and we will send out sign up info later. For now, we are targeting it to be free (including EC2 compute time), but that is predicated on us getting sponsorships to cover costs, so if you think you or your company can sponsor, please let us know ASAP.

Tentative Schedule (strawman):
8:30: Doors open/networking/coffee/snacks
9 - 9:30: Idea pitches and Seed Projects announced and teams formed — people can stand up and say what they are interested in and then we imagine people can team up based on their interest — for instance, I will probably work on Mahout and machine learning
9:30 - 12: Hack
12-1: Food/networking/hacking
1-5(?): Hack
5-6 (no firm cut off time): Share what you learned to the group over pizza and drinks. Demo if you have one.

How you can help:

- Help us get data sets organized and a Chef/Puppet recipe setup with all the appropriate tools/languages/SCM/etc. Also, think of interesting problems to work on.
- Sponsor food/coffee/drinks/t-shirts/compute time/ etc. Please contact Grant Ingersoll at info@trihug.org. I don’t think we are talking about a super lot of money here (maybe $1000-1500 total? — more on this as things develop)
- Let us know you are interested, the more we hear from sooner, the better we can plan space accordingly. Please reply on this list if you are interested.
- Are you graphically capable? Help us design a t-shirt.
- Once we firm up some details, help us spread the word

Next TriHUG: Monday April 4th

Mon, 21 Mar 2011 18:41:30 -0400

Out next meeting will be Monday, April 4th at Bronto. Food and drinks at 6:30pm. Talk starts at 7:00pm.

*** Note: this is a Monday, not our usual Tuesday ***

Title: Starfish: A Self-tuning System for Big Data Analytics

Presenter: Shivnath Babu, Duke University

Shivnath Babu, assistant professor of Computer Science at Duke University, will help demystify Hadoop performance tuning. Practical tips on tuning Hadoop for specific workloads will be discussed. Details will be provided on the the Starfish research project: a system for self-tuning big-data analytics.

RSVP HERE

February 2011 Meeting - Apache Mahout: Driving the Yellow Elephant

Wed, 19 Jan 2011 23:10:00 -0500

Out next meeting will be Tuesday February 1st at Bronto. Food and drinks at 6:30pm. Talk starts at 7:00pm.

Apache Mahout: Driving the Yellow Elephant

Apache Mahout co-founder and committer, Grant Ingersoll will give an introduction to Apache Mahout and machine learning. We will also spend some time looking at how Mahout leverages Apache Hadoop to implement a scalable clustering algorithm.

REGISTER HERE

Help spread the word! Print out this flyer and post it at your office, school, local coffee shop.

December 2010 Meeting Followup

Sat, 18 Dec 2010 07:08:17 -0500

We closed out the year with a talk by Brian O’Connor ( seen above at the MacBook Air ). He outlined how UNC Lineberger Comprehensive Cancer Center is using Hadoop and HBase in research. Once again Bronto graciously hosted us and provided food and drinks. Slides for Brian’s talk and the New and Noteworthy segment are below.

2010 was a good year for Hadoop in the Triangle. Looking forward to 2011.

-ryan

December TriHUG Meeting

Tue, 09 Nov 2010 07:39:00 -0500

We will be talking HBase and biotech. Come join us at 6:30pm on Tuesday December 7th at Bronto. Food and drinks at 6:30pm. Talk starts at 7:00pm.

Brian O’Connor from the UNC Lineberger Comprehensive Cancer Center will discuss their work using HBase and Hadoop MapReduce to store and query information from large cancer resequencing projects. He will provide an overview of HBase along with the problems they are working on. The relative merits of the technology will be explored in addition to alternative approaches.

REGISTER HERE

More info:

Help us spread the word on Twitter and Facebook or by printing out this flyer and posting it at your university or office.