[BUG] Guidely’s task descriptions and audit questions are too ambiguous

[AllenLeeyn]

Describe the bug
I have just submit my Guidely project. And I enjoyed working on it. Below are a few inconsistencies/ unclear instructions that I experienced. I hope the feedback help improve the instruction for future students.

Guidely’s task description is ambiguous and unclear in certain areas.
Guidely's Audit question contains repeated questions and unclear requirements.

Task decription:

• On Backend (FastAPI), it is stated: Endpoint to store/retrieve embeddings from the vector database. Do you mean grabbing specific chunks? or generating embedding and storing them? I assume you are not asking for us to retrieve embedding and share them with end-users.
• On the Testing & metrics, I think it will good to state clearly the intention is for students to log some of this metrics. I assume it was a suggestion that I only implemented partially.
• On the Testing & metrics, I think it will be good to split between what should be logged and what students need to keep track manually (Like Answer reference coverage require human validation)
• On the Testing & metrics: 3. Latency (local dev, warm cache), what does p95 < 5s mean?
• On the Testing & metrics: 4. Embedding cache effectiveness, I am unsure why there why will be re-embedding occur. I assume the re-embedding refer to the source data.
• On the Testing & metrics: 5. Failure handling, Is this only on the specific case stated or generic error handling?
• Bonus section comes before Dataset and code quality. One will assume what comes after Bonus is not required.

Audit questions:

• Question regarding read me appears 3 times; Preliminary, Evaluation and Metrics, Repository and Code Quality.

• Question regarding project directory setup repeated in; Preliminary, Repository and Code Quality.

• Question regarding logging of metrics repeated in; Backend (FastAPI), AI & Processing.

• I assume the project directory/ requirements.txt/ .env in the task description is a suggestion. Surprise to find it asked as a requirement here. It is a good suggestion that I followed.

• I prefer to pass the API-KEY as a system variable instead of using .env. I don't mind it being a requirement, but I think it can be stated clearly.

• In Frontend (React / Vite), Optional question Are question history or follow-up options handled (if implemented)? is there.

• In Backend (FastAPI), Optional question Is a health check or metrics endpoint implemented (optional but recommended)?is there.

• In Evaluation and Metrics: Embedding cache effectiveness (no re-embedding on unchanged files), I am unsure on how to show this. Log the reading of cache embeddings on startup?

• In Evaluation and Metrics: Failure handling (proper error messages for empty queries, corrupted data, or missing docs), is the question specifically on the few cases stated?

Users
Anyone working on this project

Severity
‼️ major

Type
🗂️ documentation
⚙️ functionality