#115 Dataclass CSV reader and Nina drops by

Sponsored by pythonbytes.fm/datadog

Special guest: Nina Zakharenko

Brian #1: Great Expectations

• A set of tools intended for batch time testing of data pipeline data.
• Introduction to the problem doc: Down with Pipeline debt / Introducing Great Expectations
• expect_[something]() methods that return json formatted descriptions of whether or not the passed in data matches your expectations.
• Can be used programmatically or interactively in a notebook. (video demo).
• For programmatic use, I’m assuming you have to put code in place to stop a pipeline stage if expectations aren’t met, and write failing json result to a log or something.
• Examples, just a few, full list is big:
  • Table shape:
    • expect_column_to_exist, expect_table_row_count_to_equal
• Missing values, unique values, and types: - expect_column_values_to_be_unique, expect_column_values_to_not_be_null
  • Sets and ranges
    • expect_column_values_to_be_in_set
  • String matching
    • expect_column_values_to_match_regex
  • Datetime and JSON parsing
  • Aggregate functions
    • expect_column_stdev_to_be_between
  • Column pairs
  • Distributional functions
    • expect_column_chisquare_test_p_value_to_be_greater_than

Nina #2: Using CircuitPython and MicroPython to write Python for wearable electronics and embedded platforms

• I’ve been playing with electronics projects as a hobby for the past two years, and a few months ago turned my attention to Python on microcontrollers
• MicroPython is a lean and efficient implementation of Python3 that can run on microcontrollers with just 256k of code space, and 16k of RAM. CircuitPython is a port of MicroPython, optimized for Adafruit devices.
• Some of the devices that run Python are as small as a quarter.
• My favorite Python hardware platform for beginners is Adafruit’s Circuit PlayGround Express. It has everything you need to get started with programming hardware without soldering. All you’ll need is alligator clips for the conductive pads.
  • The board features NeoPixel LEDs, buttons, switches, temperature, motion, and sound sensors, a tiny speaker, and lots more. You can even use it to control servos, tiny motor arms.
  • Best of all, it only costs $25.
• If you want to program the Circuit PlayGround Express with a drag-n-drop style scratch-like interface, you can use Microsoft’s MakeCode. It’s perfect for kids and you’ll find lots of examples on their site.
• Best of all, there are tons of guides for Python projects to build on their website, from making your own synthesizers, to jewelry, to silly little robots.
• Check out the repo for my Python-powered earrings, see a photo, or a demo.
• Sign up for the Adafruit Python for Microcontrollers mailing list here, or see the archives here.

Michael #3: Data class CSV reader

• Map CSV to Data Classes
• You probably know about reading CSV files
  • Maybe as tuples
  • Better with csv.DictReader
• This library is similar but maps Python 3.7’s data classes to rows of CSV files
• Includes type conversions (say string to int)
• Automatic type conversion. DataclassReader supports str, int, float, complex and datetime
• DataclassReader use the type annotation to perform validation of the data of the CSV file.
• Helps you troubleshoot issues with the data in the CSV file. DataclassReader will show exactly in which line of the CSV file contain errors.
• Extract only the data you need. It will only parse the properties defined in the dataclass
• It uses dataclass features that let you define metadata properties so the data can be parsed exactly the way you want.
• Make the code cleaner. No more extra loops to convert data to the correct type, perform validation, set default values, the DataclassReader will do all this for you
• Default fallback values, more.

Brian #4: How to Rock Python Packaging with Poetry and Briefcase

• Starts with a discussion of the packaging (for those readers that don’t listen to Python Bytes, I guess.) However, it also puts flit, pipenv, and poetry in context with each other, which is nice.
• Runs through a tutorial of how to build a pyproject.toml based project using poetry and briefcase.
• We’ve talked about Poetry before, on episode 100.
• pyproject.toml is discussed extensively on Test & Code 52.
• briefcase is new, though, it’s a project for creating standalone native applications for Mac, Windows, Linux, iOS, Android, and more.
• The tutorial also discusses using poetry directly to publish to the test-pypi server. This is a nice touch. Use the test-pypi before pushing to the real pypi. Very cool.

Nina #5: awesome-python-security *