enh(stream): fix table not exists in processTag

[wangmm0220]

Description

Please briefly describe the code changes in this pull request.

Jira: https://jira.taosdata.com:18080/browse/TD-

Checklist

Please check the items in the checklist if applicable.

• Is the user manual updated?
• Are the test cases passed and automated?
• Is there no significant decrease in test coverage?

[gemini-code-assist]

Summary of Changes

Hello @wangmm0220, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a bug in the stream processing module where an error could occur if table metadata was not properly handled during tag processing. The changes involve a significant refactoring of the tag processing mechanism, introducing a dedicated function for caching table metadata and streamlining the application of tags to data blocks. This improves the robustness and correctness of stream operations, particularly when dealing with table existence and metadata retrieval, ensuring that stream readers can correctly access table information.

Highlights

• Refactored Tag Processing: The logic for processing tags has been significantly refactored to separate the concerns of fetching and caching table metadata from the application of tags to data blocks.
• Introduced cacheTag Function: A new static function, cacheTag, has been introduced. This function is responsible for retrieving and caching table metadata (tags and table names) for a given UID, ensuring that metadata is consistently available.
• Simplified processTag: The processTag function has been simplified to now primarily retrieve already cached tag data and apply it to the data block, removing its previous responsibility for direct metadata fetching.
• Stream Table List Management API Changes: The public API for managing stream table lists has been updated. The functions qTransformStreamTableList and qStreamModifyTableList are no longer exposed in the header, and their refactored logic is now static within vnodeStream.c. New public functions qStreamSetTableList and qStreamRemoveTableList are now available for more granular control over table list modifications.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[JinqingKuang]

/gemini review

[gemini-code-assist]

Code Review

This pull request refactors stream metadata handling to fix a race condition where a table might not exist during tag processing. The core idea of pre-caching tag and schema information is a solid approach. The changes simplify the processTag function and improve consistency by centralizing SStorageAPI usage. However, I've identified a few critical issues with the implementation of the new caching mechanism, specifically related to incorrect pointer handling when using taosHashPut with isPointerType=true. These could lead to dangling pointers and memory corruption. I have also noted a minor logging issue. Addressing these points will make the solution more robust.

[JinqingKuang]

这里的 schema 包含原始表所有列，会不会有内存占用问题？

[wangmm0220]

这个schema不大，不会有问题。除非极限情况，引用了非常多的原始表，并且每个原始表列非常多。这种情况这个缓存占内存可能有点大。现在没必要为了异常极限情况做处理，处理比较麻烦。

[JinqingKuang]

后续要给这种成员变量加上统计信息，当流计算内存占用较高时，便于分析。