Getting Started with MongoDB & GDELT - Session 2 - US Session

What a blast that was!? Thanks for the invite @Shane_McAllister… awesome job @Mark_Smith… If you’re reading this and want to skip around… I put this handy time index together.

0:00 - 13:48 - Introduction and Information about the Hackathon (Mike)
13:49 - 22:10 - Introduction to GDELT Dataset
22:11 - 25:46 - Launch a MongoDB Atlas Cluster
25:47 - 27:05 - Connecting to the MongoDB Atlas Database (Obtaining the connection string)
27:06 - 27:33 - Username and password information
27:49 - 28:40 - Network security information - IP Access List Entry
30:50 - 31:55 - Installing and using python venv
32:13 - 32: 55 - gdelttools install
32:55 - 33:30 - pip tip * don’t skip this :wink:
33:59 - 34:29 - Running gdelttools - getting help, obtaining master file with gdeltloader
34:30 - 36:11 - Master file (77mb)
36:58 - 37:32 - Looking at exports - last 20 files with gdeltloader --master --download --overwrite --last 20 --filter export
38:08 - 39:49 - Looking at the data from a GDELT export in TSV format
41:05 - 41:30 - Cloning the gdelttools repo - to get some additional helpful tools
41:31 - 42:26 - Looking at the gdelt_field_file.ff - formats and field types for the GDELT data
42:27 - 42:52 - mongoimport.sh - Script to import GDELT data into mongodb
42:53 - 45:33 - Modifying mongoimport.sh to remove hard reference to database name
45:34 - 45:54 - Running mongoimport.sh - importing data into your MongoDB Atlas cluster
45:55 - 48:42 - Viewing data in Atlas - Browse Collections
48:43 - 51:30 - Optimizing mongoimport
51:31 - Transforming the imported data into improved, document-style shapes using gdelt_reshaper.js

3 Likes