We are thrilled to announce the release of the Imaged SDK, a versatile vision and NLP AI SDK, now available for iOS. Imaged SDK is a multi-platform solution supporting macOS (Intel and Apple Silicon), iOS, and Linux, with plans for Windows and Android support in the near future. This powerful SDK allows inference on CPU, CoreML, and CUDA, making it a robust choice for a variety of AI applications.

Key Features

Multi-Language Support:

• Currently Supported: C++, C, Python, Objective-C, Swift
• Planned Support: Dart, Java, Kotlin

Model Compatibility:

• Off-the-shelf models like YOLOv8 for object detection
• Popular models for object classification (CLIP-compatible), background removal, image colorizing, denoising, deblurring, interpolation, restoration, upscaling, and NSFW detection
• Specialized models for road analysis and custom on-demand models

Tutorial: Swift/iOS Integration

To demonstrate the ease of integrating Imaged SDK into your iOS project, we have prepared a tutorial showcasing the background remover feature. [Watch the video tutorial here] (Link to the video will be attached).

Steps to Use Imaged SDK for iOS:

1. Download the Framework: Download the Imaged framework from our GitHub release page.
2. Integrate the Framework: Drag and drop imaged.xcframework into your Xcode project. This framework supports both iOS and iPad, including Intel and ARM simulators.
3. Setup Bridging Header:
   • Create a new Objective-C file in your project (File > New > File [Objective C for iOS]).
   • Accept the prompt to create a bridging header file.
   • Delete the newly created Objective-C file but retain the bridging header file ${YOURPROJ}-Bridging-Header.h.
4. Import Imaged Framework: In the bridging header file, import the Imaged framework using:

   #import <imaged/sdk_objc.h>
   

5. Add Required Frameworks and Libraries: Go to General > Frameworks, Libraries, and Add the following:
   • AVFoundation.framework
   • CoreMedia.framework
   • libc++.1.tdb, libc++.tdb, and libc++abi.tdb
6. Example Swift Code: Below is an example of Swift code integrating Imaged SDK to perform background removal:

import SwiftUI

struct ContentView: View {
    @State private var originalImage: UIImage? = nil
    @State private var processedImage: UIImage? = nil
    @State private var inferenceTime: Double? = nil

    private let sdk: AISDK = {
        let sdkInstance = AISDK()
        let options = sdkInstance.getOptions()
        options.setOptionWithKey("inference.provider", stringValue: "coreml")
        options.setOptionWithKey("inference.threads.intra", intValue: 8)
        options.setOptionWithKey("inference.threads.inter", intValue: 8)
        if let path = Bundle.main.resourcePath {
            options.setOptionWithKey("rembg.model", stringValue: "\(path)/u2net.onnx")
        }
        options.setOptionWithKey("license", stringValue: "")
        return sdkInstance
    }()

    var body: some View {
        VStack {
            if let image = processedImage {
                Image(uiImage: image)
                    .resizable()
                    .scaledToFit()
                    .frame(width: 300, height: 300)
                if let inferenceTime = inferenceTime {
                    Text("Inference time: \(inferenceTime, specifier: "%.2f") ms")
                }
                Button("Reset") {
                    reset()
                }
            } else if let image = originalImage {
                Image(uiImage: image)
                    .resizable()
                    .scaledToFit()
                    .frame(width: 300, height: 300)
                Button("Remove Background") {
                    removeBackground()
                }
            } else {
                Image(systemName: "globe")
                    .imageScale(.large)
                    .foregroundStyle(.tint)
                Text("Hello, world!")
            }
        }
        .padding()
        .onAppear {
            loadImage()
        }
    }

    func loadImage() {
        guard let path = Bundle.main.resourcePath else {
            return
        }

        let image = AIImage()
        image.load(fromFile: "\(path)/car.jpg")

        if !image.isEmpty {
            self.originalImage = UIImage(contentsOfFile: "\(path)/car.jpg")
        }
    }

    func removeBackground() {
        guard let path = Bundle.main.resourcePath else {
            return
        }

        let image = AIImage()
        image.load(fromFile: "\(path)/car.jpg")

        DispatchQueue.global(qos: .userInitiated).async {
            self.sdk.load(AIModelType.rembg)

            let startTime = Date()
            self.sdk.removeBackground(image)
            let endTime = Date()

            let inferenceTime = endTime.timeIntervalSince(startTime) * 1000

            let tempDir = NSTemporaryDirectory()
            let tempFilePath = "\(tempDir)/tmp.png"
            image.save(toFile: tempFilePath)
            image.load(fromFile: tempFilePath)

            DispatchQueue.main.async {
                self.inferenceTime = inferenceTime

                if let loadedImage = UIImage(contentsOfFile: tempFilePath) {
                    self.processedImage = loadedImage
                }
            }
        }
    }

    func reset() {
        self.processedImage = nil
        self.inferenceTime = nil
    }
}

#Preview {
    ContentView()
}

Demo Notes

 

• The demo video features an image with dimensions 1200 x 632.
• Processing this image using CoreML averages a time of 133 ms.
• Processing on a CPU takes around half a second.

Get Started Today

Download the Imaged SDK and start building intelligent iOS applications with cutting-edge AI capabilities. For more information and to access the latest release, visit our GitHub page.

We look forward to seeing what you create with the Imaged SDK!