Skip to content

Commit bb17394

Browse files
regulate-techregulate-tech
authored
Merge pull request #22 from regulate-tech/minor-fixes
feat: added a step-by-step tutorial
2 parents 9a20347 + 0de2eaa commit bb17394

File tree

1 file changed

+211
-0
lines changed

1 file changed

+211
-0
lines changed

‎demo-feature-store/TUTORIAL.md‎

Lines changed: 211 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,211 @@
1+
-----
2+
***A hands-on guide to help decision makers understand technology for modern data-driven services. It is designed for non-technical audiences with support from a trainer to prepare user devices.***
3+
4+
## What is a "Feature Store"? (And Why Should You Care?)
5+
6+
**Estimated Time:** 35-50 minutes (20 mins for setup, 20 mins for the experiment)
7+
8+
Imagine you're rolling out a new NHS app. This app looks at a patient's data (age, BMI, blood pressure) and gives them a real-time diabetes risk score.
9+
10+
You have two teams:
11+
12+
1. **Your Data Scientists:** They study *months* of historical data from *all* patients to build a "risk-spotting brain" (a machine learning model).
13+
2. **Your App Developers:** They build the phone app that a GP or patient uses. It needs to get a risk score for *one* patient, right *now*.
14+
15+
Here's the million-pound question: How do you guarantee that the way the app calculates a patient's "BMI" *in real-time* is **exactly** the same way the scientists calculated "BMI" when they were training the "brain"?
16+
17+
If they are different (even slightly), the risk score could be wrong. This is a common and dangerous problem.
18+
19+
A **Feature Store** solves this.
20+
21+
Think of it as a **Central Kitchen** for your hospital's data.
22+
23+
* The kitchen prepares all the "data ingredients" (like BMI, Age, Glucose) according to a single, standard recipe.
24+
* It stores *all* historical ingredients in a giant **"Offline Freezer"** (the Offline Store) for your data scientists to study and find new patterns.
25+
* It keeps a small, fresh batch of the *latest* ingredients on a **"Online Hot Counter"** (the Online Store) for the app to grab instantly.
26+
27+
This tutorial lets you run a mini-hospital system on your own computer. You'll act as a GP, a data engineer, and a data scientist to see *why* this "central kitchen" approach is so important.
28+
29+
-----
30+
31+
## Before We Begin: Your Tools 🛠️
32+
33+
**Time:** \~5 minutes (reading)
34+
35+
In this tutorial, you'll be installing the demo software yourself. Don't worry, it's just a few copy-and-paste steps\! The goal is for you to get a hands-on feel for how these systems are pieced together.
36+
37+
To make things easy, a helper has already prepared the computer you're using with all the basic building blocks. We'll be using a few key tools:
38+
39+
1. **The Command Line (or "Terminal"):** This is the black text-based window you'll open (usually by searching for "Terminal" or "Command Prompt"). Think of it as the computer's "engine room." While your normal desktop (with icons and a mouse) is the friendly "front-of-house," the command line is where you give direct, powerful text commands to the computer's core. We'll give you the exact commands to copy and paste.
40+
41+
2. **GitHub:** This is where the project's blueprints (the code) are stored. Think of it as a "Google Docs for code." It's a website that lets teams all over the world store, share, and collaborate on their software projects. Our first command will copy the project from GitHub onto your computer.
42+
43+
3. **Python:** This is the programming language used to build the demo. Python is incredibly popular because its "grammar" (or syntax) is famously readable and clean, almost like writing in English. This makes it a top choice for everyone from app developers to data scientists.
44+
45+
4. **`pip` (Pip Installs Packages):** This is Python's "app store." It's a tool that automatically finds and installs the specific "toolkits" (or packages) our project needs, like the `Flask` web server and the `Feast` feature store.
46+
47+
With that, let's get building\!
48+
49+
-----
50+
51+
## Part 1: Setting Up Your Mini-Hospital 🏥
52+
53+
**Time:** \~10-15 minutes
54+
55+
First, we need to get the "hospital software" (the demo project) running on your computer. This involves opening your computer's "Terminal" or "Command Prompt."
56+
57+
Don't worry about what these commands mean. They are just the one-time setup steps to install the project.
58+
59+
### Step 1. Get the Project Files
60+
61+
You'll need to download the project's folder from its storage site, GitHub.
62+
63+
1. Open your Terminal (or Command Prompt).
64+
2. Copy and paste the following command to download the code, then press Enter:
65+
```bash
66+
git clone https://github.com/regulate-tech/nhstech.git
67+
```
68+
3. Now, move into the correct project folder by pasting this command and pressing Enter:
69+
```bash
70+
cd nhstech/demo-feature-store
71+
```
72+
73+
### Step 2. Create a "Clean Room" for the Project
74+
75+
This next command creates a separate, clean toolbox just for this project, so it doesn't interfere with any other software on your computer.
76+
77+
```bash
78+
python -m venv .venv
79+
```
80+
81+
### Step 3. Activate the "Clean Room"
82+
83+
Now, you "step into" that clean room.
84+
85+
* **On Windows:**
86+
```bash
87+
.venv\Scripts\activate
88+
```
89+
* **On macOS/Linux:**
90+
```bash
91+
source .venv/bin/activate
92+
```
93+
94+
You'll know it worked because you'll see `(.venv)` appear next to your terminal prompt.
95+
96+
### Step 4. Install the Tools
97+
98+
With your clean room active, you can install the "tools" this project needs (the web server, the 'AI brain' maker, etc.).
99+
100+
```bash
101+
pip install feast pandas faker scikit-learn flask joblib
102+
```
103+
104+
### Step 5. Generate Your First Patients
105+
106+
This command creates your first 500 synthetic patients and saves them to your "Offline Freezer" (a file called `patient_gp_data.parquet`) and "Online Hot Counter" (a file called `online_store.db`).
107+
108+
```bash
109+
python generate_data.py
110+
```
111+
112+
### Step 6. Train Your First "Risk Brain" 🧠
113+
114+
Now that you have data, this command tells your "data scientist" to study all 500 patients and create the first "brain" (the model) for spotting diabetes risk.
115+
116+
```bash
117+
python train_model.py
118+
```
119+
120+
### Step 7. Open the "GP's Computer"
121+
122+
You're all set\! This final command starts the simple website.
123+
124+
```bash
125+
python app.py
126+
```
127+
128+
Now, open your web browser (like Chrome or Edge) and go to this address: **`http://127.0.0.1:5000`**
129+
130+
You should see the "NHS Diabetes Risk Calculator" web page.
131+
132+
-----
133+
134+
## Part 2: The "What If" Experiment 🔬
135+
136+
**Time:** \~15-20 minutes
137+
138+
This is where you'll see the Feature Store in action. Let's run an experiment. We'll see how predictions can change for the *same patient*.
139+
140+
### Step 1. Get a Baseline
141+
142+
You're looking at the "GP View."
143+
144+
1. In the "Patient ID" box, type in **10** and click **Calculate Risk**.
145+
2. You'll see a result, something like: **Patient 10 has a 28.5% (MEDIUM RISK) chance of diabetes.**
146+
3. Note down the Patient's features (their BMI, Age, etc.) and their risk score.
147+
148+
This score is based on two things: Patient 10's data and the "Risk Brain" (model) we trained on the *first 500 patients*.
149+
150+
### Step 2. Add New National Data
151+
152+
Now, let's simulate new data arriving in the health system from across the country.
153+
154+
1. At the top of the page, click the **"Add 500 New Patients"** button.
155+
2. After the page reloads, click it **one more time**.
156+
157+
You have just added **1,000 new patients** to the "Offline Freezer" (the main database). The "Online Hot Counter" has also been updated with this new data.
158+
159+
### Step 3. See Patient 10 Again (The "Aha\!" Moment)
160+
161+
Let's check on Patient 10 again.
162+
163+
1. In the "Patient ID" box, type **10** and click **Calculate Risk**.
164+
2. You will see the **exact same risk score** as before (e.g., **28.5% - MEDIUM RISK**).
165+
166+
**Why?**
167+
This is the key insight. Their personal data hasn't changed, which is good. But more importantly, the app is still using the **old "Risk Brain"**\!
168+
169+
Even though our hospital system *has* 1,000 new patients' worth of data, our "Data Science" team hasn't published an updated "brain" yet. The app is separate from the model.
170+
171+
### Step 4. Retrain the "Risk Brain"
172+
173+
Now, let's play the role of the Data Scientist.
174+
175+
1. Click the **"Retrain Risk Model"** button.
176+
2. This tells the system: "Go to the **Offline Freezer**, study *all* 1,500 patients (the original 500 + the new 1,000), and create a *new, smarter Risk Brain* based on this much larger dataset."
177+
3. The app will automatically load this new "brain" into memory.
178+
179+
### Step 5. See the Change
180+
181+
Let's test our patient one last time.
182+
183+
1. In the "Patient ID" box, type **10** and click **Calculate Risk**.
184+
2. You will now see a **different risk score\!** (e.g., **30.1% - MEDIUM RISK**).
185+
186+
**Why?**
187+
Patient 10's personal data is *identical*. But the "brain" (the model) has been updated. It now has a more refined, more accurate understanding of risk because it learned from 1,500 patients instead of just 500.
188+
189+
This is the power of a Feature Store. The app never went offline. The GP didn't have to do anything. The "Central Kitchen" (Feature Store) ensured that the *new* model was available, and the app *instantly* started using it, providing a more accurate prediction based on the latest national data.
190+
191+
-----
192+
193+
## What You've Learned 🎓
194+
195+
You just experienced the core principle of modern data systems (often called "MLOps"):
196+
197+
* **The Problem:** In old systems, data scientists and real-time apps often use different data sources, leading to incorrect or stale predictions.
198+
* **The Solution:** A **Feature Store** acts as a "single source of truth" or "central kitchen."
199+
* The **Offline Store** ("freezer") collects all historical data for scientists to train big, new models.
200+
* The **Online Store** ("hot counter") provides the *latest* data for the live app to make instant predictions.
201+
* **The Benefit:** You can update your service's "intelligence" (the model) based on new data *without ever taking the app offline* or re-programming it. This makes your public services smarter, safer, and faster to improve.
202+
203+
-----
204+
205+
## Learn More (Drilling Down)
206+
207+
If you're curious, here are a few resources to explore these concepts further.
208+
209+
* **What is a Feature Store? (A Simple Explanation):** [A non-technical blog post on the "central kitchen" idea](https://www.google.com/search?q=https://www.tecton.ai/what-is-a-feature-store/).
210+
* **What is MLOps? (For Non-Technical People):** [An article explaining the ideas behind managing the full lifecycle of AI models](https://www.google.com/search?q=https://www.datarobot.com/blog/what-is-mlops-for-non-technical-people/).
211+
* **Feast (The Tool Used in this Demo):** [The official website for the open-source Feature Store technology used in this tutorial](https://feast.dev/).

0 commit comments

Comments
 (0)