Online Data Collection and Management

Online Data Collection and Management (oDCM)

Instructor: dr. Hannes Datta Follow @hannesdatta

Course codes: 328060-M3 (fall, block 1) and 328061-M3 (spring, block 3)

This edition: February - April 2023 | Next edition: August - October 2023

Learn how to mine the web

Welcome to the course website of oDCM.

This course teaches you the nuts and bolts about collecting web data. You learn about the technicalities of web scraping and Application Programming Interfaces (APIs), but also how to design your data collection for use in empirical (marketing) research projects1.

Please use the navigation bar and buttons below to access the course material.

Course schedule and modules Syllabus & learning objectives Enroll now!

Why should you take this course?

Ever had this fantastic research idea but didn’t have the data to study it?

Well, it’s high time you get familiar with web data gathered by web scraping and APIs! Web data is ideally suited to study questions for which commercial datasets are either unavailable or too expensive to purchase. Yes, we’re talking TikTok, Instagram, Socialblade - literally any website you can open in your browser!

Of course, you could just follow some technical tutorials on the web. But researchers face unique challenges as they select data sources for extraction, design their scrapers, and ultimately proceed to turn their prototype into a productive software tool. In this course, you will learn how to solve each of these challenges, and put your scraping and API skills into practice.

  1. Based on Boegershausen, Johannes, Hannes Datta, Abhishek Borah, and Andrew T. Stephen (2022). “Fields of Gold: Scraping Web Data for Marketing Insights.” Journal of Marketing. Download paper. Visit web companion. ↩︎