Sharding5.3.2：shadow rule error

[18835572909]

Version
Springboot 2.7.18
ShardingJdbc 5.3.2

Resource step
PartialSQLRouteExecutor#route

=> step1: sharding rule => RouteUnit(dataSourceMap<db_1,db_1>, tableMap<order_tlb,order_tlb_1>);

=> step2: shadow rule

=> ShadowSQLRouter#decorateRouteContext

	=>AbstractShadowDMLStatementRouteEngine#route
	
	=>ShadowRouteEngine#decorateRouteContext

yml

props:
sql-show: true

dataSources:
db_0:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: com.mysql.jdbc.Driver
jdbcUrl: jdbc:mysql://ip1:3306/db_0?characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=GMT%2B8&allowMultiQueries=true&allowPublicKeyRetrieval=true
username: root
password: ***
db_1:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: com.mysql.jdbc.Driver
jdbcUrl: jdbc:mysql://ip1:3306/db_1?characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=GMT%2B8&allowMultiQueries=true&allowPublicKeyRetrieval=true
username: root
password: ***
shadow_0:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: com.mysql.jdbc.Driver
jdbcUrl: jdbc:mysql://ip1:3306/shadow_0?characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=GMT%2B8&allowMultiQueries=true&allowPublicKeyRetrieval=true
username: root
password: ***
shadow_1:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: com.mysql.jdbc.Driver
jdbcUrl: jdbc:mysql://ip1:3306/shadow_1?characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=GMT%2B8&allowMultiQueries=true&allowPublicKeyRetrieval=true
username: root
password: ***

rules:

• !SHARDING
  shardingAlgorithms:
  table_inline:
  type: INLINE
  props:
  algorithm-expression: order_tbl_$->{Math.abs(order_no.hashCode()) % 2}
  database_inline:
  type: INLINE
  props:
  algorithm-expression: db_$->{Math.abs(user_id.hashCode()) % 2}
  defaultDatabaseStrategy:
  standard:
  shardingColumn: user_id
  shardingAlgorithmName: database_inline
  defaultTableStrategy:
  standard:
  shardingColumn: order_no
  shardingAlgorithmName: table_inline
  tables:
  order_tbl:
  actualDataNodes: db_$->{0..1}.order_tbl_$->{0..1}
• !SHADOW
  dataSources:
  shadowDataSource1:
  productionDataSourceName: db_1
  shadowDataSourceName: shadow_1
  tables:
  order_tbl:
  dataSourceNames:
  - shadowDataSource1
  shadowAlgorithmNames:
  - user_id_value
  shadowAlgorithms:
  user_id_value:
  type: VALUE_MATCH
  props:
  operation: select
  column: user_id
  value: x3

[terrymanu]

I have carefully analyzed the issue you reported. According to your description, ShardingSphere 5.3.2 encounters an error at the ShadowSQLRouter#decorateRouteContext step when using a combination of sharding rules and
shadow rules.

Current Understanding

Your configuration includes:

• Data sources: db_0, db_1 (production) and shadow_0, shadow_1 (shadow)
• Sharding rules: Database sharding based on user_id, table sharding based on order_no
• Shadow rules: db_1 mapped to shadow_1, using VALUE_MATCH algorithm to match user_id='x3'

The sharding rule works correctly, but it fails at the shadow rule processing step.

More Information Needed

To accurately identify the root cause, please provide the following information:

1. Complete error stack trace
   - Currently only showing method call chain, need specific exception information and complete stack trace
   - Please include exception type and error message
2. Actual executed SQL statement and parameters
   - Complete SQL statement
   - Parameter binding values (especially the actual value of user_id)
3. Simplified reproduction code
   - Minimal test code
   - Remove unnecessary business logic
4. Log configuration
   - Please confirm ShardingSphere SQL logging is enabled (you have configured sql-show: true)
   - Provide complete log output during execution

Possible Cause Analysis

Based on code analysis, the issue might occur in:

1. Data source mapping logic
   - ShadowSQLRouter needs to correctly identify whether the sharded data source is a shadow data source
   - Check the return value of findProductionDataSourceName method
2. Shadow algorithm matching
   - VALUE_MATCH algorithm needs to correctly obtain SQL parameters
   - Check if user_id parameter passing is correct
3. Configuration consistency
   - Ensure data source names in shadow rules match actual data source names produced by sharding rules

Recommended Check Steps

1. Verify shadow algorithm configuration
   shadowAlgorithms:
   user_id_value:
   type: VALUE_MATCH
   props:
   operation: select # Ensure it matches actual SQL type
   column: user_id
   value: x3
2. Check data source mapping
   - Confirm shadowDataSource1 configuration is correct
   - Verify productionDataSourceName: db_1 is consistent with sharding results
3. Enable detailed logging
   props:
   sql-show: true
   sql-simple: true

Please provide the above information, which will help us accurately identify the issue and provide a solution. If this is a ShardingSphere bug, we will fix it based on the specific situation.

[18835572909]

This picture illustrates： isShadow

this is query log

I understand： each.getDataSourceMapper().getActualName() is shading route value

but, shadowDataSourceMappings.get(actualDataSourceName); It must not be obtainable。 Is that so?

[18835572909]

this is my copy demo : https://github.com/18835572909/shadingjdbc-shadow-5.3.2.git

[terrymanu]

Could you please avoid using screenshots?
LLM cannot directly read images, and relying on screenshots may result in incomplete or inaccurate extraction of the issue details.

[18835572909]

I have tried my best to provide it, so what should I do

[terrymanu]

Please use text only, can include codes.

[18835572909]

org.apache.shardingsphere.shadow.route.engine.ShadowRouteEngine#decorateRouteContext

default void decorateRouteContext(final RouteContext routeContext, final ShadowRule shadowRule, final Map<String, String> shadowDataSourceMappings) {
    Collection<RouteUnit> toBeRemovedRouteUnit = new LinkedList<>();
    Collection<RouteUnit> toBeAddedRouteUnit = new LinkedList<>();
    for (RouteUnit each : routeContext.getRouteUnits()) {
        String logicName = each.getDataSourceMapper().getLogicName();
        String actualName = each.getDataSourceMapper().getActualName();
        Optional<String> sourceDataSourceName = shadowRule.getSourceDataSourceName(actualName);
        if (sourceDataSourceName.isPresent()) {
            String shadowDataSourceName = shadowDataSourceMappings.get(sourceDataSourceName.get());
            toBeRemovedRouteUnit.add(each);
            toBeAddedRouteUnit.add(null == shadowDataSourceName
                    ? new RouteUnit(new RouteMapper(logicName, sourceDataSourceName.get()), each.getTableMappers())
                    : new RouteUnit(new RouteMapper(logicName, shadowDataSourceName), each.getTableMappers()));
        }
    }
    routeContext.getRouteUnits().removeAll(toBeRemovedRouteUnit);
    routeContext.getRouteUnits().addAll(toBeAddedRouteUnit);
}

This is the source code:

shadowRule.getSourceDataSourceName(actualName);// 

I don't quite understand this part of him. Is the variable 'actualName' the result of shardingRule？ 

[terrymanu]

What is practiceName？

[18835572909]

oh, I wrote it wrong. Is ： actualName

[terrymanu]

After carefully analyzing your configuration and problem description, I confirm this is a design flaw in the ShardingSphere framework when handling combined sharding + shadow rule scenarios.

Root Cause

The current data source mapping logic in the ShadowRouteEngine.decorateRouteContext() method has issues:

// Current problematic code logic
String actualName = each.getDataSourceMapper().getActualName();
String sourceDataSourceName = shadowRule.getSourceDataSourceName(actualName);
String shadowDataSourceName = shadowRule.getShadowDataSourceMappings().get(sourceDataSourceName);

The problem is that the actualName after sharding routing may have been modified by the framework, causing getSourceDataSourceName() to fail to correctly reverse-resolve to the original production data source name,
ultimately resulting in shadowDataSourceName being null.

Fix Design Solutions

I suggest adopting one of the following approaches to fix this issue:

Solution 1: Enhanced Data Source Name Tracking Mechanism

// Add original data source tracking in RouteUnit
public final class RouteUnit {
private final String originalDataSourceName; // Preserve original data source name
private final DataSourceMapper dataSourceMapper;

  public RouteUnit(final String originalDataSourceName, final DataSourceMapper dataSourceMapper) {
      this.originalDataSourceName = originalDataSourceName;
      this.dataSourceMapper = dataSourceMapper;
  }

  public String getOriginalDataSourceName() {
      return originalDataSourceName;
  }

}

Solution 2: Improved Reverse Mapping Algorithm

// Enhance getSourceDataSourceName method in ShadowRule
public String getSourceDataSourceName(String routedDataSourceName) {
// Exact match
if (shadowDataSourceMappings.containsKey(routedDataSourceName)) {
return routedDataSourceName;
}

  // Fuzzy matching for sharding scenarios
  for (String dataSourceName : shadowDataSourceMappings.keySet()) {
      if (routedDataSourceName.startsWith(dataSourceName) ||
          routedDataSourceName.contains(dataSourceName)) {
          return dataSourceName;
      }
  }

  return null;

}

Solution 3: Use Route Context to Pass Mapping Relationships

// Maintain data source name mapping in RouteContext
public final class RouteContext {
private final Map<String, String> dataSourceNameMapping = new LinkedHashMap<>();

  public void addDataSourceMapping(String original, String routed) {
      dataSourceNameMapping.put(routed, original);
  }

  public String getOriginalDataSourceName(String routedName) {
      return dataSourceNameMapping.getOrDefault(routedName, routedName);
  }

}

Invitation to Participate

If you're interested in fixing this issue, we'd very much welcome you to submit a PR to complete this improvement! This is a great opportunity to contribute.

Suggested fix steps:

1. Choose one of the solutions above (I recommend Solution 1 as the clearest approach)
2. Implement the changes in your local branch
3. Write comprehensive unit tests covering various scenarios
4. Ensure existing test cases pass
5. Submit a PR and describe your fix approach

Your fix will help many developers using the combined sharding + shadow rules functionality. If you have any questions about implementation details, feel free to discuss in the PR, and we'll assist with code review and
merging.

Thank you for your contribution to the ShardingSphere project!