How to Use NCKit
Step-by-step guide for developers new to NCKit. No sample-app UI required โ copy the patterns below into your project.
Before you startโ
- Install NCKit โ iOS: confirm Embed & Sign
- iOS: physical iPhone/iPad or Apple Silicon simulator (requirements)
- Imports compile in your target
- Install NCKit and complete Android setup
- Android: API 26+ device or emulator with a supported ABI
- Imports compile in your target
Choose your use caseโ
| Goal | NCKit API | Difficulty |
|---|---|---|
| Denoise an existing audio/video file (WAV, M4A, MP4, โฆ) | NCKitFileProcessor | Easiest โ start here |
| Denoise live microphone in real time | NCKitStreamProcessor | Recommended โ sample: Audio tab |
Most apps only need file denoise (processFile). Live mic uses NCKitStreamProcessor with AVAudioEngine โ see Sample App.
| Goal | NCKit API | Difficulty |
|---|---|---|
| Denoise an existing audio/video file (WAV, M4A, MP4, โฆ) | NCKitFileProcessor | Easiest โ start here |
| Denoise live microphone in real time | NCKitStreamProcessor | Recommended โ sample: Audio tab |
| Denoise video (extract + remux UI) | NCKitFileProcessor + app MediaMuxer | Sample: Video tab |
| Denoise video/audio file (SDK only) | NCKitFileProcessor on MP4/M4A directly | Easiest โ no remux code |
Most apps only need file denoise (processFile). Real-time and video UI match Sample App.
How NCKit fits together (every integration)โ
All paths use the same two setup calls, then one processing call:
1. let modelURL = try NCKitModelLocator.modelTarGzURL()
2. let processor = try NCKitProcessor(modelURL: modelURL)
3. Process audio โ file: NCKitFileProcessor.processFile(...)
โ live: NCKitStreamProcessor.process(buffer:) (recommended)
1. val modelFile = NCKitModelLocator.modelFile(context)
2. val processor = NCKitProcessor(modelFile, attenLimDb = 100f, postFilterBeta = 0f)
3. Process audio โ file: NCKitFileProcessor.processFile(...)
โ live: NCKitStreamProcessor.process(...) (recommended)
โ live: processor.processFrame(...) (manual hops)
Create the processor once and reuse it. Creating a new NCKitProcessor per file costs ~50โ200 ms each time.
Path A โ Denoise a file (recommended first integration)โ
Step 1 โ Importโ
import NCKit
import com.fiveexceptions.nckit.*
Step 2 โ Copy-paste service (file denoise)โ
Same pattern as the NCKit Sample AudioEngine / VideoProcessor โ one shared NCKitProcessor per session.
Add this to your project. Call denoiseFile from a background thread when the user picks a file.
import NCKit
import Foundation
enum DenoiseService {
/// Reuse one processor for the app session (create on first use).
private static var processor: NCKitProcessor?
private static func processorInstance() throws -> NCKitProcessor {
if let processor { return processor }
let modelURL = try NCKitModelLocator.modelTarGzURL()
let p = try NCKitProcessor(modelURL: modelURL, attenLimDb: 100, postFilterBeta: 0)
processor = p
return p
}
/// Denoise `inputURL` and return a new 48 kHz mono WAV URL.
static func denoiseFile(inputURL: URL) async throws -> URL {
let outputURL = FileManager.default.temporaryDirectory
.appendingPathComponent(UUID().uuidString + "_clean.wav")
return try await Task.detached(priority: .userInitiated) {
let proc = try processorInstance()
try NCKitFileProcessor.processFile(
inputURL: inputURL,
outputURL: outputURL,
processor: proc
)
return outputURL
}.value
}
}
import android.content.Context
import com.fiveexceptions.nckit.NCKitFileProcessor
import com.fiveexceptions.nckit.NCKitModelLocator
import com.fiveexceptions.nckit.NCKitProcessor
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import java.io.File
object DenoiseService {
@Volatile
private var processor: NCKitProcessor? = null
private fun processorInstance(context: Context): NCKitProcessor {
processor?.let { return it }
val model = NCKitModelLocator.modelFile(context)
return NCKitProcessor(
modelFile = model,
attenLimDb = 100f,
postFilterBeta = 0f,
).also { processor = it }
}
/** Denoise [inputFile] and return a new 48 kHz mono WAV [File]. Never call on the main thread. */
suspend fun denoiseFile(context: Context, inputFile: File): File =
withContext(Dispatchers.IO) {
val outputFile = File(
context.cacheDir,
"${System.currentTimeMillis()}_clean.wav",
)
val proc = processorInstance(context)
NCKitFileProcessor.processFile(
inputFile = inputFile,
outputFile = outputFile,
processor = proc,
)
outputFile
}
}
Step 3 โ Call itโ
// Example: user picked a file (after security-scoped access if needed)
Task {
do {
let cleanURL = try await DenoiseService.denoiseFile(inputURL: pickedFileURL)
// Play, upload, or save cleanURL (16-bit PCM WAV)
} catch let error as NCKitError {
print("NCKit error: \(error)")
} catch {
print("Other error: \(error)")
}
}
// Example: user picked a file via Storage Access Framework
lifecycleScope.launch {
try {
val cleanFile = DenoiseService.denoiseFile(context, inputFile)
// Play, upload, or share cleanFile (16-bit PCM WAV)
} catch (e: NCKitException) {
Log.e("NCKit", "NCKit error", e)
} catch (e: Exception) {
Log.e("NCKit", "Other error", e)
}
}
User-selected files (important)โ
let accessed = inputURL.startAccessingSecurityScopedResource()
defer { if accessed { inputURL.stopAccessingSecurityScopedResource() } }
let cleanURL = try await DenoiseService.denoiseFile(inputURL: inputURL)
// Copy content URI to a readable File, then denoise:
contentResolver.openInputStream(uri)?.use { input ->
val temp = File(context.cacheDir, "picked_input")
temp.outputStream().use { input.copyTo(it) }
val cleanFile = DenoiseService.denoiseFile(context, temp)
}
Without proper file access, you may get cannotOpenInput / CannotOpenInput even when the file looks valid.
What you get backโ
- Input: Any format
AVAudioFilecan read (WAV, M4A, MP3, MP4, โฆ) - Output: 16-bit PCM WAV, 48 kHz, mono
- Memory: streaming โ safe for long files
- Input:
MediaExtractor-readable (WAV, M4A, MP3, MP4, โฆ) - Output: 16-bit PCM WAV, 48 kHz, mono
- Memory: streaming โ safe for long files
API details: NCKitFileProcessor
Path B โ Real-time microphone (advanced)โ
Use this when you already have (or can build) a live audio capture pipeline.
Extra requirementsโ
- NSMicrophoneUsageDescription in Info.plist
- Mic permission before starting AVAudioEngine
- NCKitStreamProcessor on the audio tap thread (not thread-safe)
- RECORD_AUDIO in AndroidManifest + runtime permission
- AudioRecord @ 48 kHz mono PCM float (recommended)
- NCKitStreamProcessor on a single capture thread (not thread-safe)
Minimal flowโ
import NCKit
import AVFoundation
// 1. Setup processor + stream adapter
let modelURL = try NCKitModelLocator.modelTarGzURL()
let processor = try NCKitProcessor(modelURL: modelURL)
let stream = NCKitStreamProcessor(processor: processor)
// 2. Configure audio session
let session = AVAudioSession.sharedInstance()
try session.setCategory(.playAndRecord, mode: .measurement, options: [.defaultToSpeaker])
try session.setPreferredSampleRate(48_000)
try session.setActive(true)
// 3. Install microphone tap
inputNode.installTap(onBus: 0, bufferSize: 4096, format: tapFormat) { buffer, _ in
try stream.prepare(inputFormat: buffer.format)
let frames = try stream.process(buffer: buffer)
for frame in frames {
// 480 Float samples @ 48 kHz mono โ denoised
}
}
Dry + denoised (A/B)โ
try stream.prepare(inputFormat: buffer.format)
let dry = try stream.convertToTargetFormat(buffer)
let denoised = try stream.processConverted(dry)
End of sessionโ
let tail = try stream.flush()
stream.reset() // requires NCKit 1.1.1+
Reference: NCKit_Demo/AudioEngine.swift. API: NCKitStreamProcessor.
import android.media.AudioFormat
import android.media.AudioRecord
import android.media.MediaRecorder
import com.fiveexceptions.nckit.NCKitModelLocator
import com.fiveexceptions.nckit.NCKitPcmFormat
import com.fiveexceptions.nckit.NCKitProcessor
import com.fiveexceptions.nckit.NCKitStreamProcessor
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
// 1. Setup processor + stream adapter (IO thread)
val processor = withContext(Dispatchers.IO) {
val modelFile = NCKitModelLocator.modelFile(context)
NCKitProcessor(modelFile, attenLimDb = 100f, postFilterBeta = 0f)
}
val stream = NCKitStreamProcessor(processor)
val format = NCKitPcmFormat(48_000, 1)
// 2. AudioRecord โ 48 kHz mono float
val sampleRate = 48_000
val channelConfig = AudioFormat.CHANNEL_IN_MONO
val encoding = AudioFormat.ENCODING_PCM_FLOAT
val minBuf = AudioRecord.getMinBufferSize(sampleRate, channelConfig, encoding)
val audioRecord = AudioRecord(
MediaRecorder.AudioSource.VOICE_COMMUNICATION,
sampleRate,
channelConfig,
encoding,
minBuf * 4,
)
audioRecord.startRecording()
// 3. Capture loop (single thread โ stream is not thread-safe)
val readBuf = FloatArray(minBuf / 4)
while (capturing) {
val n = audioRecord.read(readBuf, 0, readBuf.size, AudioRecord.READ_BLOCKING)
if (n <= 0) continue
val micSamples = readBuf.copyOf(n)
stream.prepare(format)
val frames = stream.process(micSamples, format)
for (frame in frames) {
// 480 denoised samples @ 48 kHz โ play via AudioTrack or record
}
}
// 4. End session
val tail = stream.flush()
stream.reset()
processor.close()
Dry + denoised (A/B)โ
stream.prepare(format)
val dry = stream.convertToTargetFormat(micSamples, format)
val denoised = stream.processConverted(dry)
NCKit does not set up AudioRecord / AudioTrack for you โ see Nckit_Android_Sample_App AudioEngine.kt. API details: NCKitStreamProcessor.
Direct file denoise without custom audio codeโ
// SDK decodes MP4/M4A internally โ no manual extract step required:
NCKitFileProcessor.processFile(File("input.mp4"), File("clean.wav"), processor)
API details: NCKitProcessor
Handle errorsโ
} catch NCKitError.missingModel {
// Framework not embedded correctly โ check Embed & Sign
} catch NCKitError.libraryInit {
// Intel simulator or corrupt model โ use a real device
} catch NCKitError.cannotOpenInput {
// Bad URL or missing security-scoped access
} catch NCKitError.unsupportedFormat {
// Unsupported audio format
} catch NCKitError.resampleFailed {
// AVAudioConverter failure
} catch {
print("Unexpected: \(error)")
}
} catch (_: NCKitException.MissingModel) {
// AAR not embedded โ check dependency / ProGuard
} catch (_: NCKitException.LibraryInit) {
// Corrupt model or native library failure
} catch (_: NCKitException.CannotOpenInput) {
// Bad path โ copy content Uri to temp File first
} catch (_: NCKitException.CannotCreateOutput) {
// Output directory not writable
} catch (_: NCKitException.UnsupportedFormat) {
// No decodable audio track
} catch (_: NCKitException.ResampleFailed) {
// Resampler failure
} catch (_: NCKitException.BadFrameLength) {
// Wrong hop size for processFrame
} catch (e: Exception) {
Log.e("NCKit", "Unexpected: ${e.message}")
}
Full list: NCKitError ยท Fixes: Common Errors
Common mistakes (new integrators)โ
| Mistake | Fix |
|---|---|
| Framework not Embed & Sign | Target โ General โ Frameworks |
NCKitStreamProcessor from multiple threads | Use audio tap thread only |
New NCKitProcessor per file | Reuse one instance (see DenoiseService above) |
| Intel Mac simulator | Upgrade to 1.2.1+ or use Rosetta / device |
| User-picked file won't open | Enable security-scoped resource access |
reset() missing | Upgrade to 1.2.1 (or 1.1.1+) and reset package caches |
| Denoise blocks UI | Use Task.detached for processFile |
| Expecting NCKit to build UI | NCKit is audio processing only โ UI is your app |
| Mistake | Fix |
|---|---|
processFile on main thread | Use Dispatchers.IO / background thread |
NCKitStreamProcessor from multiple threads | Use one capture thread only |
New NCKitProcessor per file | Reuse one instance (see DenoiseService above) |
| User-picked file won't open | Copy content URI to File |
UnsatisfiedLinkError | Match ABI filters: arm64-v8a, armeabi-v7a, x86_64 |
| Expecting NCKit to build UI | NCKit is audio processing only โ UI is your app |
Path C โ Denoise video audio (Android sample pattern)โ
The NCKit Sample Video tab uses this pipeline on Dispatchers.IO:
val modelFile = NCKitModelLocator.modelFile(context)
val denoisedWav = File(context.cacheDir, "denoised.wav")
NCKitProcessor(modelFile).use { processor ->
NCKitFileProcessor.processFile(originalWav, denoisedWav, processor)
}
// Optional makeup gain (sample: VideoProcessor)
NCKitAudioNormalizer.applySpeechGatedMakeupGain(samples, sampleRate = 48_000)
// Remux denoised audio back into MP4 with MediaMuxer (app-side)
NCKitFileProcessor reads any MediaExtractor-decodable file (MP4, M4A, โฆ) and writes a clean 48 kHz mono WAV. Your app handles video remux if needed.
Optional: loudness normalizationโ
import com.fiveexceptions.nckit.NCKitAudioNormalizer
val samples: FloatArray = loadWavAsFloat(denoisedFile)
NCKitAudioNormalizer.applySpeechGatedMakeupGain(
samples = samples,
sampleRate = 48_000,
targetRmsDbfs = -18f,
)
See NCKitAudioNormalizer.
API reference (lookup)โ
| Type | Page |
|---|---|
NCKitModelLocator | Model path |
NCKitProcessor | Processor |
NCKitStreamProcessor | Live audio |
NCKitFileProcessor | File denoise |
NCKitAudioNormalizer | Normalization |
NCKitError | Errors |
| Type | Page |
|---|---|
NCKitModelLocator | Model path |
NCKitProcessor | Processor |
NCKitStreamProcessor | Live audio |
NCKitFileProcessor | File denoise |
NCKitPcmFormat | PCM descriptor |
NCKitAudioNormalizer | Normalization |
NCKitException | Errors |