Difference between revisions of "Course: Big Data 2015"
Jump to navigation
Jump to search
| Line 10: | Line 10: | ||
= Background (4 weeks) = | = Background (4 weeks) = | ||
== Week 1: Course Overview; The evolution of Data Management and introduction to Big Data == | == Week 1 - Feb 2: Course Overview; The evolution of Data Management and introduction to Big Data == | ||
* Lecture notes: http://vgc.poly.edu/~juliana/courses/BigData2015/Lectures/course-overview.pdf | * Lecture notes: http://vgc.poly.edu/~juliana/courses/BigData2015/Lectures/course-overview.pdf | ||
| Line 42: | Line 42: | ||
* Programming assignment: Exploring urban data | * Programming assignment: Exploring urban data | ||
= Big Data Foundations and Infrastructure (3 weeks) = | = Big Data Foundations and Infrastructure (3 weeks) = | ||
Revision as of 16:40, 2 February 2015
DS-GA 1004- Big Data: Tentative Schedule -- subject to change
- Course Web page: http://vgc.poly.edu/~juliana/courses/BigData2015
- Instructor: Professor Juliana Freire (http://vgc.poly.edu/~juliana)
- Lecture: Mondays, 4:55pm-7:35pm at Silver, room 208.
- Some classes will include a lab session, please "always bring your laptop.
Background (4 weeks)
Week 1 - Feb 2: Course Overview; The evolution of Data Management and introduction to Big Data
- Lecture notes: http://vgc.poly.edu/~juliana/courses/BigData2015/Lectures/course-overview.pdf
- Reading: Chapter 1 of Mining of Massive Data Sets (version 1.1)
- Course survey: https://docs.google.com/forms/d/1LTiJwkDVvp0cF62Fw_d9Y86US5LCkorRUIQtV2T8KWE/viewform?usp=send_form
Week 2: Introduction to Databases, Relational Model and SQL
- Other useful reading:
Week 3: Other Data Models and Query Optimization
- Lecture notes:
- Lab: SQL
- Programming assignment: Using SQL for data analysis and cleaning
Week 4: Data Exploration and Reproducibility
- Lab: VisTrails
- Programming assignment: Exploring urban data
Big Data Foundations and Infrastructure (3 weeks)
Week 5: Cloud computing, Map Reduce and Hadoop
- Required reading:
- Data-Intensive Text Processing with MapReduce, Chapters 1 and 2
- Mining of Massive Datasets (2nd Edition), Chapter 2 - 2.1 and 2.2 (Large-Scale File Systems and Map-Reduce).
- Other useful reading:
- Hadoop: The Definitive Guide. http://www.amazon.com/Hadoop-Definitive-Guide-Tom-White/dp/1449311520
- Lab: Hands-on Hadoop
- Homework Assignment -- Your first quiz is available on Gradiance. It is due on March 17th at 5pm.
Week 6: Algorithm Design for MapReduce
- Required reading:
- Data-Intensive Text Processing with MapReduce, Chapters 1 and 2
- Mining of Massive Datasets (2nd Edition), Chapter 2.
Week 7: Parallel Databases vs MapReduce, Query Processing on Mapreduce and High-level Languages
- Lecture notes:
Big Data Algorithms, Mining Techniques, and Visualization (6 weeks)
Week 8: Visualization and Spatio-Temporal Data -- Invited lecture by Dr. Huy Vo (NYU CUSP)
- Lecture notes:
Week 9: Association Rules
- Lecture notes:
- Assignment on frequent items and association rule mining. Due on Dec 7th. Check http://www.newgradiance.com/services
- Reading: Chapter 6 Mining of Massive Datasets
- Suggested additional reading:
- Fast algorithms for mining association rules, Agrawal and Srikant, VLDB 1994.
- Data Mining Concepts and Techniques, Jiawei Han and Micheline Kamber, Morgan Kaufmann
- Dynamic Itemset Counting and Implication Rules for Market Basket Data. Brin et al., SIGMOD 1997. http://www-db.stanford.edu/~sergey/dic.html
Week 10: Finding similar items
- Reading: Chapter 3 Mining of Massive Datasets
- Homework Assignment
- There are two new quizes on Gradiance -- Distance measures and document similarity. They due on May 5th.
- Your final assignment is available at http://www.vistrails.org/index.php/Assignment_4_-_Querying_with_Pig_and_Mapreduce. This is an optional assignment and will count towards extra credit