Readme

README.md

@@ -4,135 +4,214 @@
 [![codecov](https://codecov.io/github/daviddaytw/react-native-transformers/graph/badge.svg?token=G3D0Y33SI4)](https://codecov.io/github/daviddaytw/react-native-transformers)
 [![TypeDoc](https://github.com/daviddaytw/react-native-transformers/actions/workflows/docs.yml/badge.svg)](https://daviddaytw.github.io/react-native-transformers)
 
-`react-native-transformers` is a React Native library for running Large Language Models (LLMs) from Hugging Face on your mobile applications locally. It supports both iOS and Android platforms, allowing you to leverage advanced AI models directly on your device without requiring an internet connection.
+**Run Hugging Face transformer models directly on your React Native and Expo applications with on-device inference. No cloud service required!**
 
-## Features
+## Overview
 
-- On-device transformer model support for both text generation and text embedding
-- Local inference without internet connectivity
-- Compatible with iOS and Android platforms
-- Simple API for model loading and inference
-- Support for Hugging Face models in ONNX format
-- Built on top of ONNX Runtime for efficient model execution
-- TypeScript support with full type definitions
+`react-native-transformers` empowers your mobile applications with AI capabilities by running transformer models directly on the device. This means your app can generate text, answer questions, and process language without sending data to external servers - enhancing privacy, reducing latency, and enabling offline functionality.
+
+Built on top of ONNX Runtime, this library provides a streamlined API for integrating state-of-the-art language models into your React Native and Expo applications with minimal configuration.
+
+## Key Features
+
+- **On-device inference**: Run AI models locally without requiring an internet connection
+- **Privacy-focused**: Keep user data on the device without sending it to external servers
+- **Optimized performance**: Leverages ONNX Runtime for efficient model execution on mobile CPUs
+- **Simple API**: Easy-to-use interface for model loading and inference
+- **Expo compatibility**: Works seamlessly with both Expo managed and bare workflows
 
 ## Installation
 
-To use `react-native-transformers`, you need to install `onnxruntime-react-native` as a peer dependency. Follow the steps below:
+### 1. Install peer dependencies
 
-### 1. Install the peer dependency:
+```sh
+npm install onnxruntime-react-native
+```
 
-   ```sh
-   npm install onnxruntime-react-native
-   ```
+### 2. Install react-native-transformers
 
-### 2. Install `react-native-transformers`:
+```sh
+# React-Native
+npm install react-native-transformers
 
-   ```sh
-   npm install react-native-transformers
-   ```
+# Expo
+npx expo install react-native-transformers
+```
 
-### 3. Configure React-Native or Expo
+### 3. Platform Configuration
 
 <details>
-  <summary>React Native CLI</summary>
+  <summary><b>React Native CLI</b></summary>
 
-  - Link the `onnxruntime-react-native` library:
+Link the `onnxruntime-react-native` library:
 
-    ```sh
-    npx react-native link onnxruntime-react-native
-    ```
+```sh
+npx react-native link onnxruntime-react-native
+```
 </details>
 
 <details>
-  <summary>Expo</summary>
+  <summary><b>Expo</b></summary>
 
-  - Install the Expo plugin configuration in `app.json` or `app.config.js`:
+Add the Expo plugin configuration in `app.json` or `app.config.js`:
 
-    ```json
-    {
-      "expo": {
-        "plugins": [
-          "onnxruntime-react-native"
-        ],
-      }
-    }
-    ```
+```json
+{
+  "expo": {
+    "plugins": [
+      "onnxruntime-react-native"
+    ]
+  }
+}
+```
 </details>
 
 ### 4. Babel Configuration
 
-  You need to add the `babel-plugin-transform-import-meta` plugin to your Babel configuration (e.g., `.babelrc` or `babel.config.js`):
+Add the `babel-plugin-transform-import-meta` plugin to your Babel configuration:
+
+```js
+// babel.config.js
+module.exports = {
+  // ... your existing config
+  plugins: [
+    // ... your existing plugins
+    "babel-plugin-transform-import-meta"
+  ]
+};
+```
+
+You can follow this [document](https://docs.expo.dev/versions/latest/config/babel/) to create config file, and you need to run `npx expo start --clear` to clear the Metro bundler cache.
+
+### 5. Development Client Setup
+
+For development and testing, it's required to use a development client instead of Expo Go due to the native code of ONNX Runtime and react-native-transformers.
+
+You can set up a development client using one of these methods:
+
+- **[EAS Development Build](https://docs.expo.dev/develop/development-builds/introduction/)**: Create a custom development client using EAS Build
+- **[Expo Prebuild](https://docs.expo.dev/workflow/prebuild/)**: Eject to a bare workflow to access native code
 
-   ```json
-   {
-     "plugins": ["babel-plugin-transform-import-meta"]
-   }
-   ```
 
 ## Usage
 
-### Text Generation Example
+### Text Generation
 
 ```javascript
-import React from "react";
+import React, { useState, useEffect } from "react";
 import { View, Text, Button } from "react-native";
 import { Pipeline } from "react-native-transformers";
 
 export default function App() {
-  const [output, setOutput] = React.useState("");
+  const [output, setOutput] = useState("");
+  const [isLoading, setIsLoading] = useState(false);
+  const [isModelReady, setIsModelReady] = useState(false);
+
+  // Load model on component mount
+  useEffect(() => {
+    loadModel();
+  }, []);
 
-  // Function to initialize the model
   const loadModel = async () => {
-    await Pipeline.TextGeneration.init("Felladrin/onnx-Llama-160M-Chat-v1", "onnx/decoder_model_merged.onnx");
+    setIsLoading(true);
+    try {
+      // Load a small Llama model
+      await Pipeline.TextGeneration.init(
+        "Felladrin/onnx-Llama-160M-Chat-v1",
+        "onnx/decoder_model_merged.onnx",
+        {
+          // The fetch function is required to download model files
+          fetch: async (url) => {
+            // In a real app, you might want to cache the downloaded files
+            const response = await fetch(url);
+            return response.url;
+          }
+        }
+      );
+      setIsModelReady(true);
+    } catch (error) {
+      console.error("Error loading model:", error);
+      alert("Failed to load model: " + error.message);
+    } finally {
+      setIsLoading(false);
+    }
   };
 
-  // Function to generate text
   const generateText = () => {
-    Pipeline.TextGeneration.generate("Hello world", setOutput);
+    setOutput("");
+    // Generate text from the prompt and update the UI as tokens are generated
+    Pipeline.TextGeneration.generate(
+      "Write a short poem about programming:",
+      (text) => setOutput(text)
+    );
   };
 
   return (
-    <View>
-      <Button title="Load Model" onPress={loadModel} />
-      <Button title="Generate Text" onPress={generateText} />
-      <Text>Output: {output}</Text>
+    <View style={{ padding: 20 }}>
+      <Button
+        title={isModelReady ? "Generate Text" : "Load Model"}
+        onPress={isModelReady ? generateText : loadModel}
+        disabled={isLoading}
+      />
+      <Text style={{ marginTop: 20 }}>
+        {output || "Generated text will appear here"}
+      </Text>
     </View>
   );
 }
 ```
 
-### Text Embedding Example
+### With Custom Model Download
+
+For Expo applications, use `expo-file-system` to download models with progress tracking:
 
 ```javascript
-import React from "react";
-import { View, Text, Button } from "react-native";
+import * as FileSystem from "expo-file-system";
 import { Pipeline } from "react-native-transformers";
 
-export default function App() {
-  const [embedding, setEmbedding] = React.useState([]);
+// In your model loading function
+await Pipeline.TextGeneration.init("model-repo", "model-file", {
+  fetch: async (url) => {
+    const localPath = FileSystem.cacheDirectory + url.split("/").pop();
 
-  // Function to initialize the model
-  const loadModel = async () => {
-    await Pipeline.TextEmbedding.init("Xenova/all-MiniLM-L6-v2");
-  };
+    // Check if file already exists
+    const fileInfo = await FileSystem.getInfoAsync(localPath);
+    if (fileInfo.exists) {
+      console.log("Model already downloaded, using cached version");
+      return localPath;
+    }
 
-  // Function to generate embeddings
-  const generateEmbedding = async () => {
-    const result = await Pipeline.TextEmbedding.generate("Hello world");
-    setEmbedding(result);
-  };
+    // Download file with progress tracking
+    const downloadResumable = FileSystem.createDownloadResumable(
+      url,
+      localPath,
+      {},
+      (progress) => {
+        const percentComplete = progress.totalBytesWritten / progress.totalBytesExpectedToWrite;
+        console.log(`Download progress: ${(percentComplete * 100).toFixed(1)}%`);
+      }
+    );
 
-  return (
-    <View>
-      <Button title="Load Model" onPress={loadModel} />
-      <Button title="Generate Embedding" onPress={generateEmbedding} />
-      <Text>Embedding Length: {embedding.length}</Text>
-    </View>
-  );
-}
+    const result = await downloadResumable.downloadAsync();
+    return result?.uri;
+  }
+});
 ```
 
+## Supported Models
+
+`react-native-transformers` works with ONNX-formatted models from Hugging Face. Here are some recommended models based on size and performance:
+
+| Model | Type | Size | Description |
+|-------|------|------|-------------|
+| [Felladrin/onnx-Llama-160M-Chat-v1](https://huggingface.co/Felladrin/onnx-Llama-160M-Chat-v1) | Text Generation | ~300MB | Small Llama model (160M parameters) |
+| [microsoft/Phi-3-mini-4k-instruct-onnx-web](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx-web) | Text Generation | ~1.5GB | Microsoft's Phi-3-mini model |
+| [Xenova/distilgpt2_onnx-quantized](https://huggingface.co/Xenova/distilgpt2_onnx-quantized) | Text Generation | ~165MB | Quantized DistilGPT-2 |
+| [Xenova/tiny-mamba-onnx](https://huggingface.co/Xenova/tiny-mamba-onnx) | Text Generation | ~85MB | Tiny Mamba model |
+| [Xenova/all-MiniLM-L6-v2-onnx](https://huggingface.co/Xenova/all-MiniLM-L6-v2-onnx) | Text Embedding | ~80MB | Sentence embedding model |
+
+## API Reference
+
 For detailed API documentation, please visit our [TypeDoc documentation](https://daviddaytw.github.io/react-native-transformers/).
 
 ## Contributing
@@ -154,6 +233,4 @@ This project is licensed under the MIT License. See the [LICENSE](LICENSE) file
 - [Expo Plugins Documentation](https://docs.expo.dev/guides/config-plugins/)
 - [ONNX Runtime Documentation](https://onnxruntime.ai/)
 - [Hugging Face Model Hub](https://huggingface.co/models)
-- [Babel Documentation](https://babeljs.io/)
-
-These links provide additional information on how to configure and utilize the various components used by `react-native-transformers`.
+- [ONNX Format Documentation](https://onnx.ai/onnx/intro/)

example/App.tsx

@@ -25,23 +25,39 @@ export default function App() {
     await Pipeline.TextGeneration.init(preset.model, preset.onnx_path, {
       verbose: true,
       fetch: async (url) => {
-        console.log("downloading... " + url);
-        const localpath = FileSystem.cacheDirectory + url.split("/").pop()!;
-
-        const downloadResumable = FileSystem.createDownloadResumable(
-          url,
-          localpath,
-          {},
-          ({ totalBytesWritten, totalBytesExpectedToWrite }) => {
-            setProgress(totalBytesWritten / totalBytesExpectedToWrite);
-          },
-        );
-        const result = await downloadResumable.downloadAsync();
-        if (result === undefined) {
-          throw new Error("Download failed.");
+        try {
+          console.log("Checking file... " + url);
+          const fileName = url.split("/").pop()!;
+          const localPath = FileSystem.documentDirectory + fileName;
+
+          // Check if the file already exists
+          const fileInfo = await FileSystem.getInfoAsync(localPath);
+          if (fileInfo.exists) {
+            console.log("File already exists: " + localPath);
+            return localPath;
+          }
+
+          console.log("Downloading... " + url);
+          const downloadResumable = FileSystem.createDownloadResumable(
+            url,
+            localPath,
+            {},
+            ({ totalBytesWritten, totalBytesExpectedToWrite }) => {
+              setProgress(totalBytesWritten / totalBytesExpectedToWrite);
+            }
+          );
+
+          const result = await downloadResumable.downloadAsync();
+          if (!result) {
+            throw new Error("Download failed.");
+          }
+
+          console.log("Downloaded to: " + result.uri);
+          return result.uri;
+        } catch (error) {
+          console.error("Download error:", error);
+          return null;
         }
-        console.log("downloaded as " + result.uri);
-        return result.uri;
       },
       ...preset.options,
     });