In this article we’ll parse dbt YAML into a pandas dataframe, then publish that dataframe as an interactive data grid on the web. Source code is available on GitHub and the data are available on open.quiltdata.com. Let’s get started.

Data dictionaries are a form of data documentation that help readers…

The National Hurricane Center data archive hosts hundreds of years of storm data. In search of a new dataset to explore, I pulled the Atlantic hurricane database (HURDAT2) and for a moment felt hopeless 😬. I wouldn’t call this file format cruel but it is unusual, haha.

Our file is…

In this webinar, Eric Goldbrener walks us through the highs and lows of data management for drug discovery. Eric is a consultant in compliant infrastructure for enterprises that develop drugs, targets and therapies. …

It’s common to annotate datasets with JSON. Suppose we have the following metadata:

meta_sample = {
  "author": "akarve",
  "hero_table": [
    {"name": "Thor", "powers": ["thunder", "combat"], "id": 1},
    {"name": "Iron Man", "powers": ["tech", "combat"], "id": 2},
    {"name": "Spider-Man", "powers": ["sense"…

Introduction to speakers from CorticoMetrics

In this webinar, we sit down with Lee Tirrell and Paul Wighton to discuss techniques for writing, maintaining, and debugging pipelines for quantitative neuroimaging. Lee and Paul are software engineers who solve problems in data engineering, compliance, and data science at CorticoMetrics, a company that brings quantitative neuroimaging to…

Data documentation is considerate. It forms a clue trail for anyone who wishes to analyze, reuse, or debug the data in question. There’s no shortage of recommendations on how to document data [1, 3, 4, 5, 6]. So why don’t more teams write data docs?

Because documenting data is a…

In this article you’ll learn how to create and use versioned datasets as part of a reproducible machine learning pipeline. …

Have you ever wished to reuse a code snippet, but been unable to find that code? Often, that magical snippet is hidden away in a Jupyter notebook. Mac OS Spotlight is weak at searching Jupyter notebooks. Ditto for GitHub search. …