The Future of Web AI: Integrating On-Device Models and LLMs in the Browser

The web browser is transforming into a powerful AI platform - no cloud required. From instant translations to real-time image segmentation, modern browsers now pack sophisticated AI capabilities that work offline, protect privacy, and run with native speed. Through frameworks like TensorFlow.js, Transformers.js, MediaPipe, WebLLM and Chrome's built-in AI APIs, developers can now tap into device GPUs and AI accelerated chips to run narrow and generative AI models directly in the browser. Furthermore, many open AI models are on the rise ranging from small to very large LLMs. Discover how to harness these technologies with code examples and demonstrations to build faster, more private, and cloud-independent web applications that work anywhere. The future of web AI runs in your browser - and it's ready for production today.

WebAssembly Revolution: Elevating JavaScript's Reach and Performance

In this engaging session, we'll delve into the exciting world of WebAssembly and its synergy with JavaScript. We'll explore real-world applications, such as eBay's Barcode Scanner and Photoshop Web, demonstrating Wasm's impact on performance and flexibility. Learn to integrate over 1500 npm libraries with Wasm, navigate the compilation of Wasm modules, and harness its potential outside browser environments.

Additionally, we'll look ahead at the future of Wasm and WASI, preparing you for the next wave of web development innovation. Perfect for both beginners and advanced JS developers, this talk is a must-attend for anyone eager to push the boundaries of what's possible in web apps.

Build amazing web apps with WebAssembly

Uncover the vast potential of WebAssembly (Wasm) in the realm of web development. Dive deep into its core principles while discovering practical, real-world applications where it seamlessly integrates with JavaScript. Explore how Wasm elevates web applications across diverse domains and use cases, revolutionizing the way we build for the web.

Join us on an enlightening journey into the limitless possibilities of WebAssembly. This talk promises to equip you with invaluable insights and tools to harness the true power of Wasm in your web development projects.

The Future of Web AI: Integrating On-Device Models and LLMs in the Browser

The web browser is transforming into a powerful AI platform - no cloud required. From instant translations to real-time image segmentation, modern browsers now pack sophisticated AI capabilities that work offline, protect privacy, and run with native speed. Through frameworks like TensorFlow.js, Transformers.js, MediaPipe, WebLLM and Chrome's built-in AI APIs, developers can now tap into device GPUs and AI accelerated chips to run narrow and generative AI models directly in the browser. Furthermore, many open AI models are on the rise ranging from small to very large LLMs. Discover how to harness these technologies with code examples and demonstrations to build faster, more private, and cloud-independent web applications that work anywhere. The future of web AI runs in your browser - and it's ready for production today.

Make your PWAs Look and Launch Beautifully

Building beautiful-looking icons and splash screen images for your PWA is an important aspect of the launch experience for your users. Discover the current, and upcoming, web standards to make for beautiful launch experiences, how to make them great on different platforms, and automation approaches to help you on your way toward creating stunning launch experiences for your PWAs.

This talk will be an up to date version of the talk that I only once presented at PWA Summit - https://pwasummit.org.

Previous version of the talk can be seen here: https://www.youtube.com/watch?v=gCC6gpdZnWk

Puppeteer can automate that!

Puppeteer is a node library that provides a high-level API to control Chrome over the DevTools Protocol. It is mostly used and considered for scraping content, UI testing and automated form submissions. This talk aims to change this mindset by inspiring you on observing your daily routines and common patterns on your development and testing workflows. It will help you automate them in a creative way!

We'll explore capabilities of Puppeteer with the combination of DevTools protocol and cloud functions (FaaS) across a showcase of real life use cases demonstrated over more than a dozen demos on stage. Finally, we’ll go through existing Puppeteer based SaaS solutions to inspire everyone in the room to build the next SaaS solution or open source project with Puppeteer.

I'd like to share more insights about my talk to give you the full picture. It's in markdown format, I hope you don't mind. For better readability and up-to-date info: https://hackmd.io/@onderceylan/Hyc4xF4hB

# Puppeteer can automate that!

## Description

Puppeteer is a node library that provides a high-level API to control Chrome over the DevTools Protocol. It is mostly used and considered for scraping content, UI testing and automated form submissions. This talk aims to change this mindset by inspiring you on observing your daily routines and common patterns on your development and testing workflows. It will help you automate them in a creative way!

We'll explore capabilities of Puppeteer with the combination of DevTools protocol and cloud functions (FaaS) across a showcase of real life use cases demonstrated over more than a dozen demos on stage. Finally, we’ll go through existing Puppeteer based SaaS solutions to inspire everyone in the room to build the next SaaS solution or open source project with Puppeteer.

## Technologies

My talk will touch on the grounds of web standards, graphics / css filters, performance and server side JS, cloud functions (FaaS) topics.

Following tags summarizes the tech referenced in the talk:
**chromium** **devtools** **chrome-devtools-protocol** **javascript** **nodejs** **puppeteer** **css** **test-tooling** **jest** **performance**

## Purpose of the talk

Puppeteer is most of the times used and considered by developers for scraping content or automated form submission. This is what people think about when the idea of browser automation is pitched. My goal is to change this mindset and to inspire people for the things that can be automated with a powerful browser & tooling pair. Especially, when Chrome DevTools Protocol and protocol monitor on Chrome is introduced to the audience, there's no limit on the automation possibilities.

## Target audience

Any software or QA engineer who uses Chrome / DevTools.

## Audience takeaway

I expect people to be inspired and excited by the capabilities of Puppeteer and the automation possibilities out there when I demonstrate 21 examples on stage.

Some of the demos can be seen on Twitter: [https://twitter.com/onderceylan/status/1197083395769208832](https://twitter.com/onderceylan/status/1197083395769208832)
[https://twitter.com/onderceylan/status/1202203801203232768](https://twitter.com/onderceylan/status/1202203801203232768)
[https://twitter.com/onderceylan/status/1190657108003282945](https://twitter.com/onderceylan/status/1190657108003282945)

I believe this talk will inspire everyone in the room to build the next SaaS solution or open source project with Puppeteer to automate their daily routines and to solve common problems.

## Remarks from the community

It got great feedback so far. I heard so many good things about it on offline and online channels. Some online feedbacks are like;

* [Thanks for your amazing talk Önder! You inspire us to build perf tools to our own needs.](https://twitter.com/devoorhoede/status/1197461713580363776)
* [Definitely one of the best talks !! It really inspired me to give puppeteer a try!](https://twitter.com/piyukore06/status/1197465015575547904)
* [Mind being absolutely blown at @onderceylan's talk about Puppeteer. Getting a few ideas on how I want to use it. #frontmania](https://twitter.com/rvdleun/status/1195319313265246208)
* [@onderceylan‘s talk on Puppeteer is like an easy-to-digest documentation on all the things Puppeteer can do. Will be my go to talk for anything Puppeteer for a while. @FrontMania #frontmania](https://twitter.com/PrashantPalikhe/status/1195315874921623552)
* [Inspired to automate a common development workflow after @onderceylan’s top talk on puppeteer last night!](https://twitter.com/melaniebrgr/status/1205396049139027968)
* [Awesome talk from @onderceylan on Puppeteer. Comprehensive and pragmatic. Time to replace selenium.](https://twitter.com/Root3d/status/1205213949785968641?s=20)

## Videos

Frontmania Conf Recording: https://www.youtube.com/watch?v=d2WSO3w5E94

## Source

Source of the demos and references mentioned on slides: [https://github.com/onderceylan/puppeteer-demos](https://github.com/onderceylan/puppeteer-demos)

## Slides of previous versions of the talk

* [Puppeteer can automate that! @Frontmania](https://www.slideshare.net/nderCeylan/puppeteer-can-automate-that)
* [Puppeteer can automate that! @AmsterdamJS](https://www.slideshare.net/nderCeylan/puppeteer-can-automate-that-amsterdamjs)
* [Intrducing perf budgets on CI with Puppeteer @perf.now()](https://www.slideshare.net/nderCeylan/introducing-perf-budgets-on-ci-with-puppeteer)

## Talk outline

* Introduction to headless Chrome
* Introduction to Puppeteer
* First set of demos to show some of the interesting capabilities of Puppeteer
* Introduction to Protocol Monitor and explaining how browser is being instrumented
* Introduction to Chrome DevTools Protocol
* Using CDP with Puppeteer
* Explaining how Puppeteer launches a browser instance and how one can connect to a remote instance over WS debuggable interface
* Introduction to the idea of Puppeteer Light => chrome launcher + puppeteer-core to avoid installation of a chromium bundle of over ~100mb
* More demos on utilizing CDP with Puppeteer
* Using puppeteer on cloud functions
* SaaS solutions based on Puppeteer
* Main takeaways

## Technical requirements

One of the demos I have requires sound output.

## Why me talking about Puppeteer?

I try to be the voice of developers to developers. Unlike dev rels or advocates of tech giants, I don't have to advocate for a specific product or a design decision. On the contrary, I argue the decisions taken and I introduce alternative ideas on the usage of puppeteer.

In addition, I've spent a fair amount of time on the subject on my journey of developing and maintaining my open source project: [pwa-asset-generator](https://github.com/onderceylan/pwa-asset-generator). I presented this topic in front of various communities like Frontmania, AmsterdamJS, perf.now(), C'T and more to come.