Triangle Hadoop Users Group

Slides from Feb 16. talk by Adam Gugliciello of Datameer

Wed, 22 Feb 2012 19:55:28 -0500

Thanks to Adam Gugliciello of Datameer for the talk at TriHUG last week! Here are the slides:

Financial services trihug

View more presentations from trihug.

Next Meeting: Feb. 16 @ Bronto Software

Wed, 01 Feb 2012 08:37:00 -0500

Title: Financial Data Analytics with Hadoop

Sponsored By:

RSVP here

Abstract:

Hadoop based applications are becoming critical in the financial services arena for the analysis and correlation of large volumes of structured and unstructured data. In addition, the Dodd-Frank Act signifies the largest US financial regulatory change in several decades and requires much greater transparency on financial data. In this session, we will answer common questions and demonstrate use cases in how Hadoop and Datameer help with asset management and risk management, fraud detection and data security.

Leave this session knowing about:

Financial data and Hadoop. What data lends itself to Hadoop? What doesn’t?
Benchmarks from real-world uses of Hadoop in finance
How to effectively migrate, manage, and analyze financial data using Hadoop

Bio: Adam Gugliciello, a 15-year veteran in Software Engineering and Systems Architecture specializes in highly available, parallel systems. Most recently he has been developing grid computing solutions to enable deep analyses and intelligence gathering on huge software systems for technical debt and functional mapping. Adam is a Solution Engineer at Datameer and helps bring Financial and Telco applications expertise to the utilization of the Datameer business intelligence suite.

Slides from Intro to HBase presentation January 2012

Wed, 18 Jan 2012 14:04:00 -0500

Thanks to Chris Shain from Tresata for coming to Durham last night to talk about HBase.

TriHUG January 2012 Talk by Chris Shain

Next Meeting: January 17, 2012 @ Bronto Software

Sun, 08 Jan 2012 16:54:00 -0500

Title: Intro to Apache HBase by Chris Shain of Tresata

Location: Bronto Software in Durham, NC

RSVP

Abstract: Chris will provide an introduction to Apache HBase, aiming to discuss:

What is HBase? (High level overview)
Details of the HBase architecture
How do clients interact with HBase?
Some general HBase patterns and anti-patterns
What are the use cases for HBase vs. Relational DB?

Bio: Chris Shain is the software development lead at Tresata, a provider of Big Data solutions for the financial industry in Charlotte NC. His background includes 7+ years of software development experience in the financial services industry, with a focus on customer-facing data management applications and data warehousing. Lately he works with Hadoop and HBase on data volumes in the multi-terabyte range, and tinkers with geographic information systems. He lives in Charlotte NC, and can be reached at chris@tresata.com or twitter @ChrisShain.

Slides from Alan Gates Presentation on Nov. 15, 2011

Fri, 18 Nov 2011 15:20:38 -0500

Thanks to Alan Gates of Hortonworks for the two excellent presentations on Apache Pig and Apache HCatalog. Links to the slides for the two talks are included below and are also available on Slideshare.

TriHUG November Pig Talk by Alan Gates

View more presentations from trihug.

TriHUG November HCatalog Talk by Alan Gates

View more presentations from trihug.

Slides from Oct. 11 TriHUG meeting featuring Josh Patterson of Cloudera

Wed, 12 Oct 2011 15:53:37 -0400

OSCON Data 2011 - Lumberyard View more presentations from Josh Patterson

Next Meeting: November 15, 2011 @ Bronto Software

Wed, 12 Oct 2011 08:18:00 -0400

Our next meeting will be November 15 at Bronto Software. The speaker will be Alan Gates, the author of Programming Pig and a member of the Hortonworks team. RSVP here.

————-

Title: New Features in Pig 0.9 and Introducing HCatalog

Abstract: Pig 0.9 added several features to make Pig a more powerful data processing platform, including macros, include statements, and the ability to embed Pig in Python for control flow. We’ll cover these, talk about some new features that have been added since 0.9, and what’s next on Pig’s roadmap.

HCatalog is a table management and storage management layer for Hadoop that enables users with different data processing tools – Pig, MapReduce, Hive, Streaming – to more easily read and write data on the grid. HCatalog’s table abstraction presents users with a relational view of data in the Hadoop distributed file system (HDFS) and ensures that users need not worry about where or in what format their data is stored – RCFile format, text files, sequence files. This talk will include an overview of HCatalog’s features and a discussion of its current roadmap.

Bio: Alan is a co-founder of Hortonworks as well as an original member of the engineering team that took Pig from a Yahoo! Labs research project to a successful Apache open source project. Alan also designed HCatalog and guided its adoption as an Apache Incubator project. Alan has a BS in Mathematics from Oregon State University and a MA in Theology from Fuller Theological Seminary. He is also the author of Programming Pig, a forthcoming book from O’Reilly Press. Follow Alan on Twitter: @alanfgates.

TriHUG Next Meeting featuring Josh Patterson of Cloudera set for Oct. 11

Wed, 14 Sep 2011 08:26:45 -0400

The next Triangle Hadoop User Group meeting will be October 11th at Bronto Software and will be featuring Josh Patterson of Cloudera. RSVP here.

Title: Lumberyard: Time series Indexing at Scale

Abstract:

As time series data explodes in volume in the genomic, sensor, and

financial realms [1] companies are looking for more effective ways to

store and query this data. To handle this explosion in scale systems

are looking to the Hadoop, HBase, and NoSQL domain for components to

build their systems on. In this talk we introduce Lumberyard [3], a

system which can potentially (1) store Terabytes of time series data

and allow for this data to be interactively queried at low latencies

to provide real time access. Lumberyard stores iSAX [4] indexes in

HBase’s Multi-dimensional sorted map storage system which give

Lumberyard the reliability of HDFS yet the low latencies of HBase. Our

approach leverages a multidimensional indexing structure which is

stored in HBase’s highly available distributed multi-dimensional

sorted map. We present the design of Lumberyard’s implementation and

illustrate the differences between an in-memory iSAX index compared

with a persisted HBase-backed iSAX index.

Sponsored by Cloudera and Bronto Software.

More info at www.trihug.org.

Bio:

Master’s Thesis: self-organizing mesh networks Published in IAAI-09:

TinyTermite: A Secure Routing Algorithm

Conceived, built, and led Hadoop integration for the openPDC project

at TVA (Smartgrid stuff). Led small team which designed classification

techniques for timeseries and Map Reduce. Open source work at

http://openpdc.codeplex.com

Now: Sr. Solutions Architect at Cloudera

Slides from Ted Dunning's Sept. 2011 talk

Wed, 14 Sep 2011 07:15:48 -0400

Thanks to everyone for attending last night’s talk! Ted’s slides are available for download below.

MapR, Implications for Integration View more presentations from trihug

Starfish Talk Slides from April 2011

Fri, 09 Sep 2011 08:42:13 -0400

Under the better late than never category, here are the slides from the April 2011 TriHUG meeting on Starfish.

Starfish: A Self-tuning System for Big Data Analytics View more presentations from gsingers.

Next Meeting: Sept. 13 @ Bronto Software

Sun, 28 Aug 2011 15:46:23 -0400

We trust everyone has had a good summer and is equally excited to get back into learning more about Apache Hadoop and scaling. Our next meeting will be Sept. 13 at Bronto Software. Food and drinks start at 6:30 and the talks start at 7.

We are pleased to announce that our speaker will be Ted Dunning from MapR Technologies. See below for more details. Please RSVP here.

Title: MapR, Architecture and Implications

Abstract:

The talk will be a description of how MapR’s architectural advances allow significant improvements in speed, reliability and scalability over stock Hadoop. This will include a dive into the MapR file system and a discussion of how the map-reduce layer has been changed and the impact on other Hadoop eco-system components. This will include actual test results.

In the second section of my talk, I will describe how this new architecture has surprising consequences. In particular, I will show how tasks like machine learning, data visualization and search indexing can all work better on the MapR platform.

Ted’s Bio:

Ted has held Chief Scientist positions at Veoh Networks, ID Analytics and at MusicMatch, (now Yahoo Music). Ted is responsible for building the most advanced identity theft detection system on the planet, as well as one of the largest peer-assisted video distribution systems and ground-breaking music and video recommendations systems. Ted has 15 issued and 15 pending patents and contributes to several Apache open source projects including Hadoop, Zookeeper and Hbase. He is also a committer for Apache Mahout. Ted earned a BS degree in electrical engineering from the University of Colorado; a MS degree in computer science from New Mexico State University; and a Ph.D. in computing science from Sheffield University in the United Kingdom. Ted also bought the drinks at one of the very first Hadoop User Group meetings.

RTP Scaling Hackathon (Planning Stages)

Wed, 11 May 2011 18:29:16 -0400

Some TriHUG members are in the early stage of putting together an all day hackathon on all things scaling (Hadoop, Cassandra, Hive, Pig, Mahout, etc.) and wanted to get some info out to the community as well as a call for volunteers and sponsors.

The basic gist of the day is that we get together and spend the day hacking and learning about writing scalable, fault tolerant systems. All ranges of experience are welcome and we fully expect that one of the groups that forms will be a “tutorial” group, while other groups will be doing more advanced things. The key is to get lots of interaction and cross-fertilization of ideas.

Our tentative plan is that we will make available:

1. Compute Cluster time (likely Amazon EC2) along with ready to use instances w/ appropriate things already installed. (More later)
2. Some public data sets, but feel free to bring your own publicly available on Amazon S3
3. Food, drinks, etc. including pizza/beer at the end
4. Network connectivity
5. Space to work in
6. (TBD) Machine to submit jobs using a fair scheduler

You need to bring your laptop and an open mind. Also having your favorite tools on your machine would be good. A github account or something similar would also be useful.

Our likely date for this is June 18th with a backup date of June 25 (pending space availability) from 9 AM - 6 (?) PM. Attendance will require RSVP and we will send out sign up info later. For now, we are targeting it to be free (including EC2 compute time), but that is predicated on us getting sponsorships to cover costs, so if you think you or your company can sponsor, please let us know ASAP.

Tentative Schedule (strawman):
8:30: Doors open/networking/coffee/snacks
9 - 9:30: Idea pitches and Seed Projects announced and teams formed — people can stand up and say what they are interested in and then we imagine people can team up based on their interest — for instance, I will probably work on Mahout and machine learning
9:30 - 12: Hack
12-1: Food/networking/hacking
1-5(?): Hack
5-6 (no firm cut off time): Share what you learned to the group over pizza and drinks. Demo if you have one.

How you can help:

- Help us get data sets organized and a Chef/Puppet recipe setup with all the appropriate tools/languages/SCM/etc. Also, think of interesting problems to work on.
- Sponsor food/coffee/drinks/t-shirts/compute time/ etc. Please contact Grant Ingersoll at info@trihug.org. I don’t think we are talking about a super lot of money here (maybe $1000-1500 total? — more on this as things develop)
- Let us know you are interested, the more we hear from sooner, the better we can plan space accordingly. Please reply on this list if you are interested.
- Are you graphically capable? Help us design a t-shirt.
- Once we firm up some details, help us spread the word

Next TriHUG: Monday April 4th

Mon, 21 Mar 2011 18:41:30 -0400

Out next meeting will be Monday, April 4th at Bronto. Food and drinks at 6:30pm. Talk starts at 7:00pm.

*** Note: this is a Monday, not our usual Tuesday ***

Title: Starfish: A Self-tuning System for Big Data Analytics

Presenter: Shivnath Babu, Duke University

Shivnath Babu, assistant professor of Computer Science at Duke University, will help demystify Hadoop performance tuning. Practical tips on tuning Hadoop for specific workloads will be discussed. Details will be provided on the the Starfish research project: a system for self-tuning big-data analytics.

RSVP HERE

February 2011 Meeting - Apache Mahout: Driving the Yellow Elephant

Wed, 19 Jan 2011 23:10:00 -0500

Out next meeting will be Tuesday February 1st at Bronto. Food and drinks at 6:30pm. Talk starts at 7:00pm.

Apache Mahout: Driving the Yellow Elephant

Apache Mahout co-founder and committer, Grant Ingersoll will give an introduction to Apache Mahout and machine learning. We will also spend some time looking at how Mahout leverages Apache Hadoop to implement a scalable clustering algorithm.

REGISTER HERE

Help spread the word! Print out this flyer and post it at your office, school, local coffee shop.

December 2010 Meeting Followup

Sat, 18 Dec 2010 07:08:17 -0500

We closed out the year with a talk by Brian O’Connor ( seen above at the MacBook Air ). He outlined how UNC Lineberger Comprehensive Cancer Center is using Hadoop and HBase in research. Once again Bronto graciously hosted us and provided food and drinks. Slides for Brian’s talk and the New and Noteworthy segment are below.

2010 was a good year for Hadoop in the Triangle. Looking forward to 2011.

-ryan

December TriHUG Meeting

Tue, 09 Nov 2010 07:39:00 -0500

We will be talking HBase and biotech. Come join us at 6:30pm on Tuesday December 7th at Bronto. Food and drinks at 6:30pm. Talk starts at 7:00pm.

Brian O’Connor from the UNC Lineberger Comprehensive Cancer Center will discuss their work using HBase and Hadoop MapReduce to store and query information from large cancer resequencing projects. He will provide an overview of HBase along with the problems they are working on. The relative merits of the technology will be explored in addition to alternative approaches.

REGISTER HERE

More info:

Help us spread the word on Twitter and Facebook or by printing out this flyer and posting it at your university or office.

September 2010 Meeting Followup

Wed, 22 Sep 2010 22:13:00 -0400

Another fantastic meeting. Again, it was a diverse mix from startups, big companies and academia; many of which are doing some *really* interesting stuff. Thanks again to our sponsor Lucid Imagination for the food and drinks and thanks to Bronto for the great facilities.

We have one speaker slot available for our next meeting. Interested? Hit us up @trihug.

-ryan

September TriHUG Meeting

Thu, 19 Aug 2010 07:00:00 -0400

Speakers for the next meeting are lined up. We will meet at Bronto Software, Tuesday September 14th at 6:30pm. Details in the invite link below.

REGISTER HERE

Where it all began: Using Apache Hadoop for Search with Apache Lucene and Solr

Grant Ingersoll

Apache Hadoop was originally created by Doug Cutting and Mike Cafarrela as part of the Apache Nutch project’s need for large scale crawling and indexing. In this talk, Lucene/Solr committer Grant Ingersoll will show how to use Hadoop to build a search index in Apache Lucene and Solr

Practical Hadoop Security

Wei Wei

NCSU PhD candidate Wei Wei explores attack vectors unique to distributed computing along with mitigation techniques. He will present lessons learned from his work on SecureMR: a practical service integrity assurance framework for MapReduce

Looking ahead to October, we are aiming for a biotech / pharma theme. So far we have Brian O’Connor from UNC lined up to talk about how they are using HBase in cancer research. If you want to fill the second speaker slot, head on over to the mailing list and let us know.

-ryan

July Meeting Follow-up

Thu, 22 Jul 2010 19:52:00 -0400

Our first meeting was a great success with somewhere around 25 atendees. It turns out that we have people in the Triangle using Hadoop for cancer research, quantitative finance, web analytics and a variety of other applications. Grant was furiously taking notes as ideas for future meetings were discussed. Look for announcements here on the blog, twitter and the mailing list. Thanks again to the other presenters, Jeff and Chad as well as Bronto for the facilities and jStart for the beer and pizza.

-ryan

July TriHUG Meeting

Thu, 08 Jul 2010 10:41:00 -0400

Interested in learning more about Hadoop or finding out what’s going on with Hadoop in the Triangle?

Come out for the first meeting of the Triangle Hadoop Users Group scheduled for 7pm on Tuesday July 20th at Bronto Software.

Register HERE

Three presenters are lined up:

Intro To Hadoop - Jeff Turner
Setting up Your First Cluster - Chad Vawter
Processing Megadata With Hadoop and Python - Ryan Cox

Join the mailing list and follow us @trihug.