ML5 with Webpack and TypeScript

Why TypeScript?

TypeScript allows you to add static typing to your javascript, which adds a possibility to type check on your code during compile time. Even better your IDE itself will be able to tell you if there are any issues with your data types, etc in realtime. Additionally, it can be used by your IDE for code completion. It makes you more productive by allowing you to write code quicker and also code navigation becomes very easy.

Why Webpack?

Webpack makes the build/development process for your web application easy to manage. It is extremely simple to start with and configure the basic setup and has loads of features out of the box. There are lots of advanced features like bundle optimization, code splitting, etc, and loads of plugins that can extend functions for your needs, which can be very useful in case your application is going to grow. but for the start as well webpack can work without much boilerplate code.

https://www.youtube.com/watch?v=j-h4H-NtAL8

Why ML5?

I have been playing with ML5 for some time. I have written another blog post on how to get started with ML5 and the fundamentals of Machine Learning in general. After playing for a while, I kept forgetting the methods, possible params, and their order. This led me to think…

 ..why am I not doing it in TypeScript?

I started installing all the dev dependencies. Setting up webpack, configuring eslint, and TypeScript.

There were a few ways I had to change how the sketch code worked as we are avoiding all the globals. I finally figured out. Read through or scroll down for code explanations. 

Then another problem was, although P5 already had a typedef `@types/p5`, ml5 did not have one yet and after some googling, I ended up in a discussion thread on Github https://github.com/ml5js/ml5-library/issues/253

So, there have been discussions around having TypeScript support for ml5 but the core purpose of ml5 is to be barrier-free as possible for all entry-level programmers, the TypeScript adoption, or the implementation of typedefs has not been the biggest priority for ml5.

That said, in the same thread, I found a draft from @dikarel where he has started defining few types. That seemed a very good start so I just adopted his typedefs for now and started using it, it’s not complete by any means but it’s still something.

I set up eslint for linting. The setup is now public on Github https://github.com/dejavu1987/ml5-typescript-webpack

If you want to use it you can git clone it and then install dependencies using npm install and you should be ready to start writing your Machine Learning application in no time.

Let’s explore the code.

Let’s begin with how the sketch file changed. The sketch files will be stored in the /src/ directory. You will find index.ts file with an example sketch of the MobileNet ImageClassifier.

If we were using globals, everything from p5 and ml5 is available as global variables/functions, etc. But since we don’t use them as globals anymore, we will have to import those libraries using imports like the following.

import p5 from 'p5';
import ml5 from 'ml5';

Now Sketches can not be simply created by defining global setup() and loop() function here. So, you will need to instantiate from the p5 class.

new p5((pFive) => {});

Where the callback function passed into the constructor will get the p5 object that otherwise would have been available globally. This object can be now attached with setup and loop method as follows

new p5((pFive) => {
  pFive.setup = () => {
    // Setup code here..
  };

  pFive.draw = () => {
    // Draw loop code here
  };
});

So, the general structure of a sketch will look like following

import p5 from 'p5';
import ml5 from 'ml5';

new p5((pFive) => {
  pFive.setup = () => {
    // Setup code here..
  };

  pFive.draw = () => {
    // Draw loop code here
  };
});