Skip to content

Commit c7582d3

Browse files
khanhnwinkhanhnwin
authored
Add Firebase AI Live API Plant Identifier Demo (#31)
2 parents 263daee + 53773aa commit c7582d3

File tree

113 files changed

+4276
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

113 files changed

+4276
-0
lines changed

‎firebase_ai_live_api_demo/.gitignore‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# Miscellaneous
2+
*.class
3+
*.log
4+
*.pyc
5+
*.swp
6+
.DS_Store
7+
.atom/
8+
.build/
9+
.buildlog/
10+
.history
11+
.svn/
12+
.swiftpm/
13+
migrate_working_dir/
14+
15+
# IntelliJ related
16+
*.iml
17+
*.ipr
18+
*.iws
19+
.idea/
20+
21+
# The .vscode folder contains launch configuration and tasks you configure in
22+
# VS Code which you may wish to be included in version control, so this line
23+
# is commented out by default.
24+
#.vscode/
25+
26+
# Flutter/Dart/Pub related
27+
**/doc/api/
28+
**/ios/Flutter/.last_build_id
29+
.dart_tool/
30+
.flutter-plugins
31+
.flutter-plugins-dependencies
32+
.pub-cache/
33+
.pub/
34+
/build/
35+
36+
# Symbolication related
37+
app.*.symbols
38+
39+
# Obfuscation related
40+
app.*.map.json
41+
42+
# Android Studio will place build artifacts here
43+
/android/app/debug
44+
/android/app/profile
45+
/android/app/release
46+
47+
# macOS
48+
ephemeral/
49+
50+
#firebase
51+
firebase_options.dart
52+
google-services.json
53+
GoogleService-Info.plist
54+
firebase.json

‎firebase_ai_live_api_demo/.metadata‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# This file tracks properties of this Flutter project.
2+
# Used by Flutter tool to assess capabilities and perform upgrades etc.
3+
#
4+
# This file should be version controlled and should not be manually edited.
5+
6+
version:
7+
revision: "d7b523b356d15fb81e7d340bbe52b47f93937323"
8+
channel: "stable"
9+
10+
project_type: app
11+
12+
# Tracks metadata for the flutter migrate command
13+
migration:
14+
platforms:
15+
- platform: root
16+
create_revision: d7b523b356d15fb81e7d340bbe52b47f93937323
17+
base_revision: d7b523b356d15fb81e7d340bbe52b47f93937323
18+
- platform: android
19+
create_revision: d7b523b356d15fb81e7d340bbe52b47f93937323
20+
base_revision: d7b523b356d15fb81e7d340bbe52b47f93937323
21+
- platform: ios
22+
create_revision: d7b523b356d15fb81e7d340bbe52b47f93937323
23+
base_revision: d7b523b356d15fb81e7d340bbe52b47f93937323
24+
- platform: linux
25+
create_revision: d7b523b356d15fb81e7d340bbe52b47f93937323
26+
base_revision: d7b523b356d15fb81e7d340bbe52b47f93937323
27+
- platform: macos
28+
create_revision: d7b523b356d15fb81e7d340bbe52b47f93937323
29+
base_revision: d7b523b356d15fb81e7d340bbe52b47f93937323
30+
- platform: web
31+
create_revision: d7b523b356d15fb81e7d340bbe52b47f93937323
32+
base_revision: d7b523b356d15fb81e7d340bbe52b47f93937323
33+
- platform: windows
34+
create_revision: d7b523b356d15fb81e7d340bbe52b47f93937323
35+
base_revision: d7b523b356d15fb81e7d340bbe52b47f93937323
36+
37+
# User provided section
38+
39+
# List of Local paths (relative to this file) that should be
40+
# ignored by the migrate tool.
41+
#
42+
# Files that are not part of the templates will be ignored by default.
43+
unmanaged_files:
44+
- 'lib/main.dart'
45+
- 'ios/Runner.xcodeproj/project.pbxproj'

‎firebase_ai_live_api_demo/README.md‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# Firebase AI Live API Demo
2+
**Target Platforms:** iOS, Android, Web
3+
4+
**Tech Stack:** [Flutter](https://flutter.dev/) (frontend), [Firebase AI Logic](https://firebase.google.com/docs/ai-logic) (Gemini API in Vertex AI for the backend)
5+
6+
![Plant Identifier – Firebase AI Live API Demo App on mobile & web](README/PlantIdentifierHero.png)
7+
8+
This app demonstrates how to build a Flutter app with real-time bidirectional
9+
audio & video streaming to the Gemini "Live API" using Firebase AI Logic.
10+
11+
As seen in as seen in the
12+
["What's New in Flutter"](https://youtu.be/v6Rzo5khNE8?si=0316B2O7xDM4Zp4S&t=2278)
13+
Google I/O 2025 keynote.
14+
15+
## Getting Started
16+
17+
1. Follow [these instructions](https://firebase.google.com/docs/ai-logic/get-started?&api=vertex#set-up-firebase)
18+
to set up a Firebase project
19+
20+
1. Connect the app to your Firebase project by using `flutterfire configure`.
21+
22+
Install `flutterfire_cli`:
23+
24+
```console
25+
flutter pub global activate flutterfire_cli
26+
```
27+
28+
Then run the `flutterfire` command to configure this project for your Firebase project:
29+
30+
```console
31+
rm lib/firebase_options.dart
32+
flutterfire configure
33+
```
34+
35+
1. Run `flutter pub get` in the root of the project directory `firebase_ai_live_api_demo` to
36+
install the Flutter app dependencies
37+
38+
1. Run `flutter run -d <device-id>` to start the app on iOS, Android, or Web.
39+
40+
> [!TIP]
41+
> Get available devices by running `flutter devices` ex: `AA8A7357`, `macos`, `chrome`.
42+
> The live video functionality won't work on iOS simulators due to camera restrictions.
43+
44+
## How to use the demo app
45+
46+
1. When prompted, allow the app permission to access your camera and microphone.
47+
48+
1. Click the call button and say "Hello Gemini!"
49+
50+
2. Turn on your device camera by clicking the camera button.
51+
52+
1. You'll see in `lib/src/flutterfire_ai_live_api_demo.dart` that the app
53+
is pre-configured with a "plant identifier" system instruction. So point your camera at a plant and ask Gemini to identify it!
54+
55+
```dart
56+
systemInstruction: Content.text(
57+
'You are a plant identifier. Greet the user by telling them that you '
58+
'are a plant identifier. Ask them to turn on their camera and show '
59+
'you a plant and you can help them identify plants and flowers. '
60+
'Your job is to help the user dentify plants and flowers. '
61+
'When the user asks you to identify a plant or flower, respond '
62+
'by telling them what it is and along with fun fact about it. '
63+
'If you\'re unable to identify the plant or flower, you may ask the user '
64+
'for more information about it or ask for a closer look.',
65+
)
66+
```
67+
68+
If you have some time on your hands, try to modify the system instruction,
69+
model configuration, or add tools that let Gemini retrieve real-time info
70+
or to take take some sort of action.
71+
72+
## Additional Resources
73+
- [[Codelab] Build a Gemini powered Flutter app with Flutter & Firebase AI Logic](https://codelabs.developers.google.com/codelabs/flutter-gemini-colorist)
74+
- [Firebase AI Logic docs](https://firebase.google.com/docs/ai-logic)
75+
76+
Feeling inspired? Check out these other Flutter & Firebase AI Logic sample apps!
77+
- [Agentic App Manager](https://github.com/flutter/demos/tree/main/agentic_app_manager): Build an agentic experience in a Flutter app using Firebase AI Logic with the Gemini API in Vertex AI.
78+
- [Colorist](https://github.com/flutter/demos/tree/main/vertex_ai_firebase_flutter_app): Explore LLM tooling interfaces by allowing users to describe colors in natural language. The app uses Gemini LLM to interpret descriptions and change the color of a displayed square by calling specialized color tools.

‎firebase_ai_live_api_demo/README/PlantIdentifierHero.png‎

1.97 MB
Loading

‎firebase_ai_live_api_demo/analysis_options.yaml‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# This file configures the analyzer, which statically analyzes Dart code to
2+
# check for errors, warnings, and lints.
3+
#
4+
# The issues identified by the analyzer are surfaced in the UI of Dart-enabled
5+
# IDEs (https://dart.dev/tools#ides-and-editors). The analyzer can also be
6+
# invoked from the command line by running `flutter analyze`.
7+
8+
# The following line activates a set of recommended lints for Flutter apps,
9+
# packages, and plugins designed to encourage good coding practices.
10+
include: package:flutter_lints/flutter.yaml
11+
12+
linter:
13+
# The lint rules applied to this project can be customized in the
14+
# section below to disable rules from the `package:flutter_lints/flutter.yaml`
15+
# included above or to enable additional rules. A list of all available lints
16+
# and their documentation is published at https://dart.dev/lints.
17+
#
18+
# Instead of disabling a lint rule for the entire project in the
19+
# section below, it can also be suppressed for a single line of code
20+
# or a specific dart file by using the `// ignore: name_of_lint` and
21+
# `// ignore_for_file: name_of_lint` syntax on the line or in the file
22+
# producing the lint.
23+
rules:
24+
# avoid_print: false # Uncomment to disable the `avoid_print` rule
25+
# prefer_single_quotes: true # Uncomment to enable the `prefer_single_quotes` rule
26+
27+
# Additional information about this file can be found at
28+
# https://dart.dev/guides/language/analysis-options

‎firebase_ai_live_api_demo/android/.gitignore‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
gradle-wrapper.jar
2+
/.gradle
3+
/captures/
4+
/gradlew
5+
/gradlew.bat
6+
/local.properties
7+
GeneratedPluginRegistrant.java
8+
.cxx/
9+
10+
# Remember to never publicly share your keystore.
11+
# See https://flutter.dev/to/reference-keystore
12+
key.properties
13+
**/*.keystore
14+
**/*.jks

‎firebase_ai_live_api_demo/android/app/build.gradle.kts‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
plugins {
2+
id("com.android.application")
3+
// START: FlutterFire Configuration
4+
id("com.google.gms.google-services")
5+
// END: FlutterFire Configuration
6+
id("kotlin-android")
7+
// The Flutter Gradle Plugin must be applied after the Android and Kotlin Gradle plugins.
8+
id("dev.flutter.flutter-gradle-plugin")
9+
}
10+
11+
android {
12+
namespace = "com.example.flutterfire_ai_live_api_demo"
13+
compileSdk = flutter.compileSdkVersion
14+
ndkVersion = flutter.ndkVersion
15+
16+
compileOptions {
17+
sourceCompatibility = JavaVersion.VERSION_11
18+
targetCompatibility = JavaVersion.VERSION_11
19+
}
20+
21+
kotlinOptions {
22+
jvmTarget = JavaVersion.VERSION_11.toString()
23+
}
24+
25+
defaultConfig {
26+
// TODO: Specify your own unique Application ID (https://developer.android.com/studio/build/application-id.html).
27+
applicationId = "com.example.flutterfire_ai_live_api_demo"
28+
// You can update the following values to match your application needs.
29+
// For more information, see: https://flutter.dev/to/review-gradle-config.
30+
minSdk = 23
31+
targetSdk = flutter.targetSdkVersion
32+
versionCode = flutter.versionCode
33+
versionName = flutter.versionName
34+
ndkVersion = "27.0.12077973"
35+
}
36+
37+
buildTypes {
38+
release {
39+
// TODO: Add your own signing config for the release build.
40+
// Signing with the debug keys for now, so `flutter run --release` works.
41+
signingConfig = signingConfigs.getByName("debug")
42+
}
43+
}
44+
}
45+
46+
flutter {
47+
source = "../.."
48+
}

‎firebase_ai_live_api_demo/android/app/src/debug/AndroidManifest.xml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
2+
<!-- The INTERNET permission is required for development. Specifically,
3+
the Flutter tool needs it to communicate with the running application
4+
to allow setting breakpoints, to provide hot reload, etc.
5+
-->
6+
<uses-permission android:name="android.permission.INTERNET"/>
7+
</manifest>

‎firebase_ai_live_api_demo/android/app/src/main/AndroidManifest.xml‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
2+
<application
3+
android:label="flutterfire_ai_live_api_demo"
4+
android:name="${applicationName}"
5+
android:icon="@mipmap/ic_launcher">
6+
<activity
7+
android:name=".MainActivity"
8+
android:exported="true"
9+
android:launchMode="singleTop"
10+
android:taskAffinity=""
11+
android:theme="@style/LaunchTheme"
12+
android:configChanges="orientation|keyboardHidden|keyboard|screenSize|smallestScreenSize|locale|layoutDirection|fontScale|screenLayout|density|uiMode"
13+
android:hardwareAccelerated="true"
14+
android:windowSoftInputMode="adjustResize">
15+
<!-- Specifies an Android theme to apply to this Activity as soon as
16+
the Android process has started. This theme is visible to the user
17+
while the Flutter UI initializes. After that, this theme continues
18+
to determine the Window background behind the Flutter UI. -->
19+
<meta-data
20+
android:name="io.flutter.embedding.android.NormalTheme"
21+
android:resource="@style/NormalTheme"
22+
/>
23+
<intent-filter>
24+
<action android:name="android.intent.action.MAIN"/>
25+
<category android:name="android.intent.category.LAUNCHER"/>
26+
</intent-filter>
27+
</activity>
28+
<!-- Don't delete the meta-data below.
29+
This is used by the Flutter tool to generate GeneratedPluginRegistrant.java -->
30+
<meta-data
31+
android:name="flutterEmbedding"
32+
android:value="2" />
33+
</application>
34+
<!-- Required to query activities that can process text, see:
35+
https://developer.android.com/training/package-visibility and
36+
https://developer.android.com/reference/android/content/Intent#ACTION_PROCESS_TEXT.
37+
38+
In particular, this is used by the Flutter engine in io.flutter.plugin.text.ProcessTextPlugin. -->
39+
<queries>
40+
<intent>
41+
<action android:name="android.intent.action.PROCESS_TEXT"/>
42+
<data android:mimeType="text/plain"/>
43+
</intent>
44+
</queries>
45+
</manifest>

‎firebase_ai_live_api_demo/android/app/src/main/kotlin/com/example/multimodal_ai_prototype/MainActivity.kt‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
package com.example.flutterfire_ai_live_api_demo
2+
3+
import io.flutter.embedding.android.FlutterActivity
4+
5+
class MainActivity : FlutterActivity()

0 commit comments

Comments
 (0)