Voice & Audio AI
No Sound Library: 12 Programmatic Sound Recipes and a Mixing Standard
Natural sounds come from only two raw ingredients: filtered noise and decaying vibration. Every sound can be redone by changing one line of parameters.
Who this is for: anyone making short films who needs ambience and music but wants no licensing trouble.
Core idea: natural sounds have two raw ingredients
| Ingredient | How to make it | What it makes |
|---|---|---|
| Filtered noise | Generate white noise from random numbers, keep one frequency band with a filter, apply a volume envelope (attack, decay) | Rain, wind, waves, footsteps, brush strokes, splashes |
| Decaying vibration | A sine wave (or a few stacked), with volume decaying exponentially | Strings, bells, drums, water drops |
Plain-language glossary:
- White noise: the “shhh” where every frequency is equally loud.
- Filter: lets only one range of pitch through. Low-pass keeps the lows, high-pass keeps the highs, band-pass keeps a middle slice.
- Volume envelope: the loudness curve of a sound from appearing, to swelling, to fading.
12 sound recipes
| Sound | Recipe |
|---|---|
| Rain | Two noise layers: a 500–7000Hz “hiss” plus a 150–700Hz “rumble”; then sprinkle random 1800–6000Hz sine “pings” that decay in 4 ms as raindrops. Rain intensity is just a volume curve over time |
| Thunder | Brown noise (noise accumulated over time, very deep), low-pass at 180Hz, slow attack, 6 second decay; add 0.15 seconds of high-frequency noise at the start as the “crack” |
| Muddy footsteps | A 60–90Hz sine thump decaying in 60 ms, plus a band-pass noise sweeping up from 300Hz to 2000Hz (the squelch), then a short downward-sweeping suction sound 0.3 seconds later |
| Wading | A downward-sweeping band-pass noise (splash), low-frequency noise wobbling around 350Hz (water sloshing), and a few small rising sine slides (droplets) |
| Wooden clogs | Two sines at 1650Hz and 2750Hz decaying in 20 ms, plus a very short noise burst as the strike |
| Water drop | A sine whose pitch slides up within 40 ms (400 to 1200Hz) while decaying quickly |
| Brush stroke | 2000–5000Hz band-pass noise, fading in and out within 0.5 seconds |
| Zither (guzheng) | Karplus-Strong: a short burst of noise of length sample rate divided by pitch, looped, averaging each pair of adjacent samples and multiplying by 0.996 every pass |
| Shakuhachi | Fundamental sine plus a little 2nd and 3rd harmonic, plus band-pass breath noise; pitch slides up from a semitone below at the attack; add a 5 Hz vibrato after 0.6 seconds |
| Temple bell | Partial ratios 0.5, 1, 1.19, 1.51, 2, 2.51, 2.66, 3.01, 4.17 (not integer multiples), low ones decaying 8 to 12 seconds and high ones 1 to 2 seconds; give each of the lowest two a “twin” 0.4–0.6% higher in frequency to create beating |
| Taiko | A 55Hz sine that starts slightly higher and drops, 0.4 second decay, plus a hit of low-passed noise |
| Insects | A 4300–4800Hz sine switched on and off about 28 times a second for the “chirp”, sounding only in the first third of each second |
Reverb (for a sense of space): a simplified Freeverb, 8 comb filters plus 4 all-pass filters. Raise the mix for night and indoor scenes.
A prompt for an AI (swap in any sound):
Write a function in Node.js (no packages) that produces a mono audio array of [sound name] at a 44100 sample rate. Recipe: [paste the table cell]. Finally mix several sounds at their time points and export a 16-bit stereo WAV file.
Keep sound and picture permanently in sync
Write “when the character walks and how long each step takes” as one piece of data, for example:
{"id": "walk-1", "start": 9.3, "end": 22.6, "period": 0.62, "surface": "mud"}
- The animation computes the character’s stride and body bob from this data.
- The sound effects use the same data to place a footstep at each step’s moment.
- When the story changes, edit only this data and both sides recompute, so they always line up.
Mixing standard (the minimum bar before delivery)
- Duck automatically when a voice is present: music down to about 38%, ambience about 28%, foley about 55%.
- Measure voice clarity above 200Hz: the voice should be 8.5 dB or more above the background. A full-band measurement gets fooled by the low end of the rain.
- 40Hz high-pass on the master, to remove inaudible sub-bass that still eats headroom.
- Soft limiting plus loudness normalization to about -16 LUFS (ffmpeg
loudnorm=I=-16:TP=-1.5). - Check the structure: use a spectrogram to see whether the rain’s rise and fall, the silent points and the bell positions are right.
ffmpeg -i track.wav -lavfi showspectrumpic=s=1600x500 spectrum.png
Emotional arrangement cheat sheet
- Opening: one bell strike and a few plucked notes to establish stillness.
- Walking: rain plus footsteps, sparse melody.
- Key character enters: bring in a new timbre for the first time, such as shakuhachi.
- Conflict: one taiko hit.
- Before the thesis line: let all ambience go quiet for 3 seconds, then strike the bell. The sound after silence carries the most weight.
Back to the start
Not one of the rain, the temple bell or the zither was recorded, and every one can be redone by changing a line of parameters.
This is part of an ongoing series of field notes and AI tool tests. To get the full SOPs and future posts, subscribe to the newsletter.
Support this work：Support on Ko-fi