James Graham

Software / Games / Web

My Projects

Click on a project!

Reform Artwork

Lua

Renoise API

The #1 most-liked Renoise tool

Reform

Reform is a granular MIDI-editing tool for Renoise. It is the #1 top tool in the Renoise Tool Forums.

Write-Up

The Concept

After the launch of Paddles, the need for a new Renoise tool became apparent; one that would make it quick and intuitive to powerfully fine–tune strums. The idea was inspired by FL Studio's "Strumizer" tool. It offers knobs and sliders to edit timings and velocities. You get realtime visual feedback of your changes to the notes. You can preview the audio by pressing the spacebar. Renoise, in comparison, lacks any such abilities, requiring users to edit notes one–by–one instead.

FL Studio's Strumizer

FL Studio's Strumizer

The Problem

Being a tracker, Renoise represents musical notes as text on a spreadsheet. By default, each cell on the spreadsheet corresponds to a 16th note. To have notes trigger in–between this 16th–note grid, you must type a hexadecimal value 0x00–0x80 into the "delay" column next to the note. This results in an inefficient process for creating strums that are not in sync with the grid's time division, requiring the user to manually type individual hexadecimal values for each note.

Manual Strum Editing in Renoise

Strum editing in Renoise

The Solution

A strum–editing tool for Renoise that would include: a playful UI, intuitive controls, realtime updates to notes, easy preview playback, strumming across pattern boundaries, and more. This would be quite a complex task to achieve in a tracker–based application.

The Development

Though it would take much too long to discuss every challenge and triumph of Reform's development here, we will discuss a few of the noteworthy ones.

Caching

To allow manipulated notes to overflow into other patterns, avoid colliding with other notes, and wrap from the end of the song to the beginning, it was necessary to know pattern lengths, song lengths, note positions, and more. The handing-over of all that data from Renoise's C++ runtime to its Lua API is slow, and Reform demands huge amounts of that data rapidly, hundreds of times per second. To improve performance, the data would be cached Lua–side after retrieving it from the Renoise runtime, and Renoise's Observer–style notifiers would be used to alert Reform when cached data was no longer valid. This resulted in a dramatic increase in performance (over 100x), and was architected in an easy-to-use, modular API.

Caching Diagram

Basic Caching Diagram

Curving

The implementation of Beziér curves in Reform (developed via the Curves demo) allowed for selections of notes to have their timings "bent". Note timings are interpreted as being linearly distributed upon selection, and can then be redistributed along a Quadratic Beziér curve, or an S-shaped Cubic curve. More curve shapes could easily be added, and users could even be allowed to create custom curves via an editor, but for the majority of use–cases, these two curve types are sufficient.

Reform's Curve Function Demo

Reform's Curve function in-action

The Result

The tool is accurate, performant, and fully–featured to a professional standard. This was achieved through long–term project organization, disciplined learning, patient problem–solving, and clever optimization. The tool has been provided for free, out of deep appreciation for the Renoise software.

Reform Demo

Demonstrating some of Reform's functions

Paddles Demo

Lua

Renoise API

Game for Renoise

Paddles

Paddles is a game written in Lua, that was met with praise upon its release, and is currently featured on the front page of Renoise's Tools website.

Write-Up

The Concept

Being a musician (drums, synths), one of my favorite programs is Renoise. After learning to program, I remembered that Renoise offers a Lua API to create extensions of the program! I decided to make a game within Renoise. Wanting something classic, I decided on Pong. The only other game made using the Renoise API is a port of Nibbles ("Nibbles" is a Snake clone included in FastTracker2).

Nibbles Gameplay

Renoise port of "Nibbles"

The Development

Learning Lua

I learned Lua using the official guide and documentation. I found my favorite feature of the language to be tables. Tables are so simple in contrast to C++'s structs, arrays, enums, vectors, hashmaps, etc. If you understand how they work under the hood, they can be very performant, too!

Engineering a Display

The Renoise API doesn't really offer a way to directly control pixels. Instead, it offers elements like sliders, knobs, text boxes, etc. However, I found that I could control a grid of tiny bitmaps (loaded from the filesystem) to create a "virtual screen"! Each frame, I had to manually update bitmaps, rather than blanking the whole screen and redrawing everything as graphics engines typically do, because Renoise isn't very fast at updating large amounts of bitmaps (it can update ~100 bitmaps at 25 frames/second).

Renoise Controls

Renoise ViewBuilder API controls & my bitmaps

The Launch

After some development, the game had a rainbow that trailed behind the ball, an AI opponent, a 2-player mode, sound effects, and more! Within 24 hours of its release, it was featured on the front page of the Renoise website's "Tools" section (where it still remains today), and even received recognition from Renoise's lead programmer and CEO; Eduard Müller (aka "taktik").

Early Paddles Demo

Early release of Paddles

The Update

Upgrading the Display

After the game's release, I discovered that Renoise provided another GUI element capable of creating a "virtual screen". Buttons can be scaled, and colored via a HEX color-code. This change offered a full 8-bit color display—capable of displaying 16,777,216 colors! To top it off, Renoise updates them 9x faster than bitmaps! I also implemented a double frame buffer system that only pushed updates to buttons that needed it. This allowed for more complex graphics, as I could now blank the screen each frame!

Button Demonstration

A button

Utilizing the New Display

I had been studying some water simulation algorithms for use in my 3D graphics projects, and decided to implement a water simulation in my Pong game for some nice visuals! The update was released and received more positive feedback from the Renoise community!

Paddles V3.0 Demo

Latest release of Paddles

Reflecting

If I were to continue working on this project, I would enhance the sound effects, and add a Breakout-style game mode. I would also like to write a 3D renderer for Renoise's Lua API someday, and create a small 3D game using it, though other projects are currently higher priorites.

Hopes

I sometimes worry that Renoise will cease development. Its keyboard–only tracker-style approach was common from the 1980's to the early 2000's, but most users today prefer a piano–roll interface. I hope that my Renoise tools will attract more like-minded users to the program, and keep its development alive for years to come.

Piano Roll vs. Tracker

The same MIDI sequence in a piano roll and a tracker

Curves Demo

Lua

Renoise API

Bézier Curves

Curves

Curves is a pixel-perfect, weighted, infinite-degree Bézier curve editor written in Lua, created as a testing ground for a feature being developed for Reform.

Write-Up

The Purpose

During development on Reform, the need arose for the calculation and rasterization of various curves. My first solution was to use logarithmic functions. They worked, but implementing discrete mathematical functions for different curve shapes wasn't a very robust solution. I ended up going down a rabbit hole of math and 2D rasterization that yielded the desired result. This project was the testing ground where I isolated this task.

The Development

Infinite-Degree Bézier Curves

After a lot of digging, I found Bézier Curves. Compared to Linear, Quadratic, and Cubic Bézier curves, I wanted my curves to support an infinite number of control points. I found this recursive definition of a Bézier curve. After implementing this—along with the associated functions for Binomial Coefficients and Bernstein Basis Polynomials—my curves supported an infinite number of control points! A very proud moment indeed!

Curves of Varying Degrees

Curves of varying degrees

Tension

I wanted each control point to have a variable "weight" (or "tension") to control its influence on the curve. I found an amazing resource called A Primer on Bézier Curves by Mike Kamermans (aka "Pomax"), which contains a chapter on adding this "weight"! After implementing this new math, I now had infinite-degree, weighted, Bézier curves!

Curve Tension

Adjusting weight

Rasterization

Bézier curves are rendered in a unique way. Rather than solving for X / Y coordinates per–pixel, you have to solve for t—which represents an interpolation between the curve's endpoints.

Showing t interpolation

t being interpolated

Naiveté

A naive approach is to just sample a high number of points, and fill the pixels where those points lie. The result of this approach is a curve that's too thick in high–tension segments, and breaking apart in low–tension segments. A better approach was clearly needed.

Curve sample sizes

Uneven t distribution

Pixel-Perfect

My solution was to sample a moderate amount of points, and connect them with line segments, which are easier to rasterize (which I later discovered is called "flattening" the curve). More robust solutions have been documented (such as in Chapter 4 of A Rasterizing Algorithm for Drawing Curves by Alois Zingl), but this solution would be sufficient, and quicker to implement. Wanting pixel–perfect, aliased lines, I found the Bresenham algorithm, which yielded a beautiful result!

Flattening the curve

Flattening a curve

Finished!

Being modular/loosely coupled, my finished code was easily implemented into Reform, with 2 curve types available now, and a custom curve editor to come in a future update. I will definitely be utilizing the knowledge from this project in other future projects as well! I can already imagine the possibilities of what can be created using these techniques!

Reform Curve Demo

Bézier curves implementation in Reform

SNEK Demo

C++

WinCon.h

FMOD

ASCII game for the Windows Console

SNEK

SNEK is an ASCII-rendered game for the Windows console, and uses the FMOD Studio framework for procedural sound effects, and beat-synced music playback.

Write-Up

Starting Out

This was my first substantial programming project. I chose C++ as my first programming language because I wanted a low–level foundation for my understanding of programming, and because I find working with computers at a low level to be fun and interesting (for these reasons, I've also dabbled in GBZ80 assembly, and the circuitry of GameBoy modding). I chose the Windows Console for my first project because it has a very simple API that would allow me to focus on learning fundamental programming concepts.

Rendering

The game was first rendered using cout statements, but this didn't result in a framerate–capable runtime (although it did have a cool film–like effect). So I learned and refactored my code to write directly to the console buffer using the WriteConsoleOutputCharacter() function, which resulted in a stable image capable of faster framerates. It also allowed me to easily add some color to the game, using the WriteConsoleOutputAttribute() function.

SNEK Old Version vs New Version

Rendering comparison: cout vs WriteConsoleOutput()

Sound

As the project matured, I wanted to add sound to the game, and decided to learn FMOD Studio and the FMOD Studio API. From this endeavor I learned how to read API documentation, which assured me that I would be able to learn and use any libraries I want for future projects. I did successfully add sound effects, as well as music that is synced to the player's movement (each frame that the snake moves corresponds to an 8th note in the music). The music also reacts when the player eats a fruit exactly on a downbeat, gets a new high score, and more.

Music from the game
(Song sections progress as
the player's score increases)

Reflecting

Now that I've matured as a developer, I can see there are areas in this project where code can be cleaned up, and more elegant solutions can be implemented for certain features. But, I've decided to leave it as–is for the time being, and learn new technologies with more potential for a wider impact. If I were to return to this project, my first priority would be to make it cross–platform with Windows, Linux, and Mac. Then I would add more gameplay features and visual effects.

3D Graphics Demo

C#

HLSL

Unity

3D Graphics Studies

3D Graphics Studies

Some 3D graphics studies centered around water rendering. Everything shown was modeled and rendered from scratch in Blender and Unity's ShaderGraph (along with some custom HLSL). Water mesh vertex displacement done in C#.

HTML

CSS

JS

Three.js

Portfolio (this website)

Portfolio

This is my portfolio website, created with pure HTML5, CSS3, and JavaScript, with a little bit of Three.js.

Write-Up

Design

I love 3D graphics. My favorite 3D effect is refraction, particularly when used to create water. I thought an interactive water effect would make my portfolio website a lot of fun to browse.

Adaptive Performance

To accomodate a wide range of devices, Three.js's rendering resolution gets reduced on-the-fly if the current framerate is too low. This allows for an HD experience on high-end devices, while still maintaining a smooth framerate on lower-end devices.

Adaptive Geometry

The Quadratic formula is used to ensure the water's geometry has a consistent density of vertices regardless of the aspect ratio of the HTML document. This means the water simulation moves at the same speed regardless of where or how the website is being viewed.

About Me

Picture of me!

James Graham

I'm developing audio & music tools, and video games (primarily in C++)!

The focus of my studies is realtime audio (MIDI, DSP) and realtime graphics (2D, 3D). I also have some experience in web development, UI design, and I learned quite a bit about text encodings (ASCII, UTF-8, UTF-16, UTF-32) while working on my console/terminal game development library, "ConArtist".

Aside from programming, I'm a lifelong musician (drummer, percussionist, sound designer, composer, producer), and I'm also practicing visual art (pixel art, 3D modeling, watercolor). As a result, I'm quite experienced with many modern audio & visual design softwares. I enjoy seeing art from both sides; as a creative, and as an engineer.

Check out my Github, and contact me on LinkedIn, or by email!

Skills

Languages

C++

Experience: 5+ Years

My favorite language! I've used C++ to write audio DSP code for Godot, create games and libraries for the Win32 console, and to solve LeetCode problems. Currently working on an embedded project in C++.

Lua

Experience: 3+ Years

I became very comfortable using Lua while creating tools & games for Renoise (an audio/music software). I learned many under-the-hood details about the language through the official Lua documentation.

JavaScript

Experience: 1+ Years

I learned JavaScript while creating this website! I'm continuing to learn the language as I do web development freelance work and update this website.

CSS

Experience: 1+ Years

I learned CSS while creating this website! I'm continuing to learn the language as I do web development freelance work and update this website.

HTML

Experience: 1+ Years

I learned HTML while creating this website! I'm continuing to learn the language as I do web development freelance work and update this website.

GBZ80
Assembly

Experience: 2+ Months

I learned the basics of Assembly for the Nintendo GameBoy using RGBDS. I created a couple of Hello World programs that display text and animated sprites.

Frameworks

Three.js

Experience: 1+ Years

I used Three.js to create the 3D background for this website! By dynamically altering the normals of a refractive mesh in Three.js, I created the water/ripple effect you see here!

FMOD

Experience: 2+ Years

I used FMOD Studio and its C++ API to implement audio in my terminal game, "SNEK".

Godot

Experience: 2+ Years

I wrote audio DSP code for Godot in November of 2022 (PR #68768)! I'm also using the engine to work on some game prototypes.

Unity

Experience: 3+ Years

The first game engine I learned, it led me to learn C# and discover programming! I created 3D shaders using ShaderGraph and HLSL, and wrote some basic gameplay protoypes in C#.

Arduino

Experience: 2+ Years

I'm using the Arduino environment (Teensy 4.0) to get prototypes for my current project up and running, but will be moving to a more capable processor/custom board down the line.

PlatformIO

Experience: 9+ Months

I've been using PlatformIO + VS Code to upload my C++ code to the Teensy 4.0 for my current project.

Tools

Git

Experience: 4+ Years

I'm very comfortable with Git. Multiple branches, rebasing, amending, bare repos... I'm not a master by any means, but I'm quite comfortable with Git.

Visual
Studio

Experience: 5+ Years

I've used Visual Studio for C++ code. I'm comfortable using the profiler, debugger, configuring Project & Solution settings, and more.

VS Code

Experience: 1+ Years

I use VS Code to write all of my HTML, CSS, and JavaScript. I like learning new hotkeys! (P.S. I'm typing this in VS Code right now!)

Notepad++

Experience: 10+ Years

First used to edit Minecraft server files over 10 years ago. Now use it to open source files for a quick peek or to make small edits.

webpack

Experience: 3+ Months

I use webpack to bundle Three.js for this website. It's much more reliable and fast-loading than including Three.js from a CDN!

npm

Experience: 3+ Months

Used in tandem with webpack to bundle Three.js for this website!

Design

Blender

Experience: 4+ Years

I use Blender to create all of my own 3D models for my website, games, and visual artwork.

Photoshop

Experience: 10+ Years

I've used Photoshop over the years to create pixel art, edit my 3D art, create stickers and printable magnets, and more!

Aseprite

Experience: 3+ Years

I've been using Aseprite to create pixel art instead of Photoshop ever since I discovered the program!

Renoise

Experience: 9+ Years

My favorite audio/music software! I could go on and on and on and on and on... just try this program. It's so good.

FL Studio

Experience: 11+ Years

My other favorite audio/music software! Harmor is incredible.

LSDJ

Experience: 8+ Years

Music software that runs on a Nintendo Game Boy. I run up to 4 systems in sync for more polyphony!

Links