You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/on-device-ai-goes-mainstream.mdx
+15-15Lines changed: 15 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -33,7 +33,7 @@ Two megatrends are converging:
33
33
34
34
-**[Edge Computing](https://objectbox.io/dev-how-to/edge-computing-state-2025)** - Processing data where it is created, on the device, locally, at the edge of the network, is called "Edge Computing" and it is growing
35
35
-**AI** - AI capabilities and use are expanding rapidly and without a need for further explanation
36
-
<imgsrc="/img/edge-ai/edge-ai.png"alt="Edge AI: Where Edge Computing and AI intersect" />
36
+
<imgsrc="img/edge-ai/edge-ai.png"alt="Edge AI: Where Edge Computing and AI intersect" />
37
37
38
38
--> where these two trends overlap (at the intersection), it is called Edge AI (or local AI, on-device AI, or with regards to a subsection: "Mobile AI")
39
39
@@ -45,18 +45,18 @@ The shift to Edge AI is driven by use cases that:
45
45
* are not economically viable when using the cloud / a cloud AI
46
46
* want to be sustainable
47
47
48
-
<imgsrc="/img/edge-ai/edge-ai-benefits.png"alt="Edge AI drivers (benefits)" />
48
+
<imgsrc="img/edge-ai/edge-ai-benefits.png"alt="Edge AI drivers (benefits)" />
49
49
50
50
If you're interested in the sustainability aspect, see also: [Why Edge Computing matters for a sustainable future](https://objectbox.io/why-do-we-need-edge-computing-for-a-sustainable-future/)
51
51
52
52
## It's not Edge AI vs. Cloud AI - the reality is Hybrid AI
53
53
54
54
Of course, while we see a market shift towards Edge Computing, there is no Edge Computing vs. Cloud Computing - the two complement each other and the question is mainly: How much edge does your use case need?
55
55
56
-
<imgsrc="/img/edge-ai/cloud-to-edge-continuum.png"alt="Edge AI drivers (benefits)" />
56
+
<imgsrc="img/edge-ai/cloud-to-edge-continuum.png"alt="Edge AI drivers (benefits)" />
57
57
58
58
Every shift in computing is empowered by core technologies
59
-
<imgsrc="/img/edge-ai/computing-shifts-empowered-by-core-tech.png"alt="Every shift in computing is empowered by core technologies" />
59
+
<imgsrc="img/edge-ai/computing-shifts-empowered-by-core-tech.png"alt="Every shift in computing is empowered by core technologies" />
60
60
61
61
## What are the core technologies empowering Edge AI?
62
62
@@ -67,7 +67,7 @@ Typically, Mobile AI apps need **three core components**:
67
67
2. A [**vector database**](https://objectbox.io/vector-database/)
68
68
3.**Data sync** for hybrid architectures ([Data Sync Alternatives](https://objectbox.io/data-sync-alternatives-offline-vs-online-solutions/))
@@ -76,15 +76,15 @@ Typically, Mobile AI apps need **three core components**:
76
76
77
77
Large foundation models (LLMs) remain costly and centralized. In contrast, **Small Language Models (SLMs)** bring similar capabilities in a lightweight, resource-efficient way.
78
78
79
-
<imgsrc="/img/edge-ai/slm-quality-cost.png"alt="SLM quality and cost comparison" />
79
+
<imgsrc="img/edge-ai/slm-quality-cost.png"alt="SLM quality and cost comparison" />
80
80
- Up to **100x cheaper** to run
81
81
- Faster, with lower energy consumption
82
82
- Near-Large-Model quality in some cases
83
83
84
84
This makes them ideal for **local AI** scenarios: assistants, semantic search, or multimodal apps running directly on-device. However....
85
85
86
86
### Frontier AI Models are still getting bigger and costs are skyrocketing
87
-
<imgsrc="/img/edge-ai/llm-costs-still-skyrocketing.png"alt="SLM quality and cost comparison" />
87
+
<imgsrc="img/edge-ai/llm-costs-still-skyrocketing.png"alt="SLM quality and cost comparison" />
88
88
89
89
### Why this matters for developers: Monetary and hidden costs of using Cloud AI
90
90
@@ -99,10 +99,10 @@ Running cloud AI comes at a cost:
99
99
### What about Open Source AI Models?
100
100
101
101
Yes, they are an option, but be mindful of potential risks and caveats. Be aware that you also pay to be free of liability risks.
102
-
<imgsrc="/img/edge-ai/opensource-ai-models.png"alt="SLM quality and cost comparison" />
102
+
<imgsrc="img/edge-ai/opensource-ai-models.png"alt="SLM quality and cost comparison" />
103
103
104
104
### While SLMs are all the rage, it's really about specialised AI models in Edge AI (at this moment...)
105
-
<imgsrc="/img/edge-ai/for-mobile-it-is-specialized-models-not-SLM.png"alt="SLM quality and cost comparison" />
105
+
<imgsrc="img/edge-ai/for-mobile-it-is-specialized-models-not-SLM.png"alt="SLM quality and cost comparison" />
106
106
107
107
108
108
## On-device Vector Databases are the second essential piece of the Edge AI Tech Stack
@@ -118,7 +118,7 @@ Edge Vector databases, or on-device vector databases, are still rare. Some serve
## Developer Story: On-device AI Screenshot Searcher Example App
@@ -133,19 +133,19 @@ To test the waters, I built a [**Screenshot Searcher** app with ObjectBox Vector
133
133
This was easy and took less than a day. However, I learned more with the stuff I tried that wasn't easy... ;)
134
134
135
135
### What I learned about text classification (and hopefully helps you)
136
-
<imgsrc="/img/edge-ai/on-device-text-classification.png"alt="On-device Text Classification Learnings" />
136
+
<imgsrc="img/edge-ai/on-device-text-classification.png"alt="On-device Text Classification Learnings" />
137
137
138
138
--> See Finetuning.... without Finetuning, no model, no text classification.
139
139
140
140
### What I learned about finetuning (and hopefully helps you)
141
-
<imgsrc="/img/edge-ai/finetuning-text-model-learnings.png"alt="Finetuning Learnings (exemplary, based on finetuning DBpedia)" />
141
+
<imgsrc="img/edge-ai/finetuning-text-model-learnings.png"alt="Finetuning Learnings (exemplary, based on finetuning DBpedia)" />
142
142
143
143
--> Finetuning failed --> I will try again ;)
144
144
145
145
### What I learned about integrating an SLM (Google's Gemma)
146
146
147
147
Integrating Gemma was super straightforward; it worked on-device in less than an hour (just don't try to use the Android emulator (AVD) - it's not recommended to try and run Gemma on the AVD, and it also did not work for me).
148
-
<imgsrc="/img/edge-ai/using-gemma-on-android.png"alt="Using Gemma on Android" />
148
+
<imgsrc="img/edge-ai/using-gemma-on-android.png"alt="Using Gemma on Android" />
149
149
150
150
151
151
In this example app, we are using Gemma to enhance the screenshot search with an additional AI layer:
@@ -161,7 +161,7 @@ It's already fairly easy - and vibe coding an Edge AI app very doable. While of
0 commit comments