Mastering Pandas: An Intermediate Course in Data Analysis and Manipulation

This two-day course is designed to provide intermediate-level training in data science with a focus on the pandas library. Pandas is a powerful open-source data analysis and manipulation tool built on top of the Python programming language.

Throughout the course, attendees will learn advanced techniques for manipulating and visualizing data with pandas, including how to work with large and complex datasets, optimize performance, and handle text and time series data.

By the end of this course, attendees will have a strong foundation in data science with pandas and will be able to use this powerful tool to perform a wide range of real-world data analysis tasks.

Agenda:

Day 1:
- Introduction to Pandas
- Series and DataFrames
- Indexing and selecting data
- Reading from files and other input sources
- Handling missing values
- Basic operations and statistical methods
- Manipulating Data
- Merging, joining, and concatenating data
- Grouping and pivot tables
- Reshaping and pivoting data
- Working with time series data

Day 2:
- Visualizing Data with Pandas
- Line plots, scatter plots, and bar plots
- Histograms and density plots
- Box plots and violin plots
- Heatmaps and pair plots
- Advanced Pandas Techniques
- Handling large and complex datasets
- Performance and optimization tips
- Working with text data and categorical data
- Advanced indexing and reshaping techniques
- Advanced groupby and aggregation operations

This course is designed to be accessible to those with no previous knowledge of the Python programming language. However, having some familiarity with programming concepts will be helpful.

Attendees should bring a laptop with python installed (3.7 or above).

Hands-on with MicroPython

Embedded programming is more accessible than ever. In this workshop we will get hands-on with MicroPython, a Python implementation for microcontrollers.

We will cover the basics of the MicroPython language and the provided microcontroller. We will connect sensors, joysticks, buttons and displays and make them work for us.

Run code examples on pre-assembled hardware or design your own creations. The important thing is to have fun!

Data Frames - Getting close and personal with your dataset

A data frame is a data structure specifically designed for effective management of tabular data, conceptually similar to a spreadsheet or a database table. Originating from the statistical computing language R, it has since gained widespread adoption in the data science community with Spark as well as the Pandas Python package.

Working knowledge of data frames and how to use them is helpful for any developer, as it provides an easy way to analyze, process and visualize data from a multitude of sources.

In this session we will be exploring some everyday use case scenarios for data frames, using Pandas and Jupyter.

IoT is fun using MicroPython

Python might not be a language you would expect to see running on an embedded device. Yet here we are. That time has come. You can now run a Python REPL session directly on your microchip..

MicroPython is a Python implementation for embedded processors. It provides you with a subset of Python 3.5 features including numeric types, strings, tuples, lists, dictionaries and classes. Large parts of the Python standard library have also been ported.

This session will give an introduction to MicroPython, discuss how it differs from traditional CPython, and how to get up and running with coding yourself.

Rapid Prototyping for IoT

An NGO needed to detect when their remote water tanks were running low, so that they could provide more reliable access to drinking water. We designed a sensor system that sends alerts via SMS when the water level drops. Two evenings later, a working prototype was built and on its way to Sierra Leone for testing. It cost less than a Norwegian beer, and the code spans less than 100 lines.

Rapid prototyping such as this water level sensor is made possible by using high-level programming languages on IoT devices. Languages such as Python and Javascript are surprisingly effective for creating simple and even complex programs for microcontrollers and other embedded systems. No flashing required, simply upload your .py or .js files to the controller over USB or WiFi.

Sensors, displays and input devices have become cheap and readily available through mainstream retailers, and drivers for these devices are easy to come by, owing to a vibrant open source community. It’s now easy to make your own weather station, garage door opener, mp3 player or a toy for the kids.

In this workshop, participants are given a custom made game controller packed with hardware components such as displays, buttons, joysticks and sensors. Drivers and example code for each component will be provided. We will use MicroPython, an easy-to-learn language for developers of all backgrounds. Interactive workshop exercises may include:

- Gathering sensor data and pushing it to the cloud
- Coding an autonomous robot car
- Writing a multiplayer game using the game controllers

MicroPython is a lean and efficient version of Python, rewritten in C and specifically designed for constrained systems. It comes with a complete parser, compiler, virtual machine, runtime system and garbage collector. Uploaded source code is compiled dynamically on the chip itself. MicroPython is quickly gaining popularity in parts of the industry and is already being run on commercial products.

We will cover
- The powerful ESP32 microcontroller
- Embedded vs normal systems

- Getting started
- Initial, one-time flashing of the MicroPython firmware
- Uploading your code to the microcontroller
- Connecting to the interactive shell
- Using built-in python modules

- Interfacing with your hardware components
- Writing your first program
- Task scheduling (preemptive, cooperative)
- Internet connectivity and messaging
- Strategies for code optimization

From circuit board design to finished product: the hobbyist’s guide to hardware manufacturing

Ever wondered how hardware is made, or curious about making your own?

In this session, we will share our experiences with building a programmable gamepad for use in IoT workshops. The gamepad is equipped with:

- WiFi and Bluetooth
- Touch screen
- Microphone and speaker
- Joypad and buttons
- Integrated sensors

We will cover the entire production process, including:

- Designing the PCB (Printed Circuit Board)
- Choosing a microcontroller and parts
- Finding, ordering and assembling components
- Pulling together firmware, drivers and software

We conclude with live, interactive demos using the gamepad.

Harmful Business Patterns in Software Consulting

Customer lock-in. Dubious “Land and Expand” strategies. What do you do when the interests of your employer are at odds with those of your client? Perhaps a taboo subject, but one worth talking about.

You might be encouraged to promote over-complicated solutions, so you can stock up with consulting resources. Maybe you are expected to recommend an inferior in-house product over better, or open-source alternatives.

As a software consultant, you are expected to provide the best possible value proposition to your customer. But you also have loyalty to your employer.

In this session we look into the various ethical dilemmas arising from such conflicts of interests, and some approaches to combat them.

Data Frames - It's Excel for programmers

Load your sales data from csv/json/db. Filter out transactions under 10 USD. Aggregate by city. Make a choropleth map and show your boss that we are underselling in New York. This is what ten lines of code can give you.

A data frame is a data structure specifically designed for effective management of tabular data, conceptually similar to a spreadsheet or a database table.

Working knowledge of data frames and how to use them is helpful for any developer, as it provides an easy way to analyze, process and visualize data from a multitude of sources.

In this session we will be exploring some everyday use case scenarios for data frames, using Pandas and Jupyter.