Summary

Any time that you are storing data about people there are a number of privacy and security considerations that come with it. Privacy engineering is a growing field in data management that focuses on how to protect attributes of personal data so that the containing datasets can be shared safely. In this episode Gretel co-founder and CTO John Myers explains how they are building tools for data engineers and analysts to incorporate privacy engineering techniques into their workflows and validate the safety of their data against re-identification attacks.

Announcements

• Hello and welcome to the Data Engineering Podcast, the show about modern data management
• When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm. With simple pricing, fast networking, object storage, and worldwide data centers, you’ve got everything you need to run a bulletproof data platform. Go to dataengineeringpodcast.com/linode today and get a $100 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
• Atlan is a collaborative workspace for data-driven teams, like Github for engineering or Figma for design teams. By acting as a virtual hub for data assets ranging from tables and dashboards to SQL snippets & code, Atlan enables teams to create a single source of truth for all their data assets, and collaborate across the modern data stack through deep integrations with tools like Snowflake, Slack, Looker and more. Go to dataengineeringpodcast.com/atlan today and sign up for a free trial. If you’re a data engineering podcast listener, you get credits worth $3000 on an annual subscription
• Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. By the time errors have made their way into production, it’s often too late and damage is done. Datafold built automated regression testing to help data and analytics engineers deal with data quality in their pull requests. Datafold shows how a change in SQL code affects your data, both on a statistical level and down to individual rows and values before it gets merged to production. No more shipping and praying, you can now know exactly what will change in your database! Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.
• TimescaleDB, from your friends at Timescale, is the leading open-source relational database with support for time-series data. Time-series data is time stamped so you can measure how a system is changing. Time-series data is relentless and requires a database like TimescaleDB with speed and petabyte-scale. Understand the past, monitor the present, and predict the future. That’s Timescale. Visit them today at dataengineeringpodcast.com/timescale
• Your host is Tobias Macey and today I’m interviewing John Myers about privacy engineering and use cases for synthetic data

Interview

• Introduction
• How did you get involved in the area of data management?
• Can you describe what Gretel is and the story behind it?
• How do you define "privacy engineering"?
  • In an organization or data team, who is typically responsible for privacy engineering?
• How would you characterize the current state of the art and adoption for privacy engineering?
• Who are the target users of Gretel and how does that inform the features and design of the product?
• What are the stages of the data lifecycle where Gretel is used?
• Can you describe a typical workflow for integrating Gretel into data pipelines for business analytics or ML model training?
• How is the Gretel platform implemented?
  • How have the design and goals of the system changed or evolved since you started working on it?
• What are some of the nuances of synthetic data generation or masking that data engineers/data analysts need to be aware of as they start using Gretel?
• What are the most interesting, innovative, or unexpected ways that you have seen Gretel used?
• What are the most interesting, unexpected, or challenging lessons that you have learned while working on Gretel?
• When is Gretel the wrong choice?
• What do you have planned for the future of Gretel?

Contact Info

• LinkedIn
• @jtm_tech on Twitter

Parting Question

• From your perspective, what is the biggest gap in the tooling or technology for data management today?

Closing Announcements

• Thank you for listening! Don’t forget to check out our other show, Podcast.__init__ to learn about the Python language, its community, and the innovative ways it is being used.
• Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
• If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com) with your story.
• To help other people find the show please leave a review on iTunes and tell your friends and co-workers

Links

• Gretel
• Privacy Engineering
• Weights and Biases
• Red Team/Blue Team
• Generative Adversarial Network
• Capture The Flag in application security
• CVE == Common Vulnerabilities and Exposures
• Machine Learning Cold Start Problem
• Faker
• Mockaroo
• Kaggle
• Sentry

The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA