Bug: Different capacity numbers

[peterdudfield]

make doctor output

(Im not sure what this is)

Verification

• I have searched the existing issues to ensure this bug has not already been reported.
• I have run make init locally.
• I have been able to reproduce this bug locally.

What were you trying to do (and why)?

• I did not make init as working with PVLive consumer

Update the capacity for a gsp location.

Python logs save this

grpclib.exceptions.GRPCError: (<Status.INVALID_ARGUMENT: 3>, 'Invalid location source. Ensure metadata is NULL or a non-empty JSON object.', None)
You might want to install geopandas

Serve logs say this

�[2Kdata-platform  | 
{
    "level": "error",
    "grpc.method": "/ocf.dp.DataPlatformDataService/UpdateLocationCapacity",
    "error": "ERROR: duplicate key value violates unique constraint \"sources_history_pkey\" (SQLSTATE 23505)",
    "time": "2025-11-13T17:25:31Z",
    "message": "querier.CreateSourceEntry({GeometryUuid:019a7231-3570-7568-b1d8-00af5edefc7d SourceTypeID:1 Capacity:1429 CapacityUnitPrefixFactor:3 CapacityLimitSip:<nil> ValidFromUtc:{Time:2025-11-13 17:00:00 +0000 UTC InfinityModifier:finite Valid:true} Metadata:[123 34 103 115 112 95 105 100 34 58 32 51 52 50 125]})"

What happened (including relevant commands, logs etc)?

Tried to update the capacity of location, but cants

What did you expect to happen?

No updated capacity was needed

Step-by-step reproduction instructions (optional).

1. Update the capacity to 1398373
2. Read the new capacity as 1398000. This is what it is saved as in the database
3. Then next time PVLive consumer runs, it tried to update the capacity again, it cant due to unqieu primary keys.

Note if we run 3. after 30 mins it is ok.

[peterdudfield]

Other examples are
444000000 to 444234720

[peterdudfield]

This maybe becasue capacity_unit_prefix_factor is set to 3, but im not sure

[peterdudfield]

Looks like 39MW is also a problem, this has capacity_unit_prefix_factor set to 6

[peterdudfield]

I wonder if a solution, is to not go 3,6,9,12 in the capacity_unit_prefix_factor, but jump in 3,4,5,6...12 then we might get more arracuary.

I might need reminding why we dont just save the raw value, something about smallint being more database efficient

[devsjc]

The reason we don't save the raw value is as you say - more efficient to save values of variying scales. Postgres BIGINT maxes out at 10^14, which is "only" one order of magnitude higher than the current worldwide renewable generation. Maybe that is enough headroom and we could convert; in the case that it isn't (and I like to be ambitious here) you have two options:

• use kw as the base unit (but that prevents us from accurately representing small household solar installations that may only be around a kilowatt)
• store the number and its magnitude seperately

I ruled out the first one, for the reason above. The second can be achieved with the NUMERIC type, but calculations on it are slower in postgres than integers or floats. As such I arrived at the current solution - saving the exponent seperately. to the value, both with SMALLINTS.

The reason they are saved in stages of 3 are to mirror the increments of the SI prefixes. This was to reflect what I expected capacity values to be given in: fractions of an SI unit (1.245MW, 35.6kW, etc). I hadn't considered that for a site measured in megawatts, we would be interested in the capacity down to a single watt - e.g. someone is saying 1.398373MW - seems very precise to me, but I'm not on the client end of things so I don't know what is usual here.

As it stands, in the expected case of people measuring capacity at a specific SI scale, we have 5 significant figures of accuracy. In the case of tracking all indices as you suggest, we have 5 significant figures at all times, even if someone is not measuring nicely at a prefix.

I guess my question is, is the source of inaccuracy from truncating capacities down to 5 s.f. on the produced forecasts really going to be large, when compared to the inaccuracy of the forecasts themselves? How many significant figures do we produce our forecast estimates to? Are they really going to meaningfully affected by movement in the fifth decimal place? Is it even reasonable for us to say our forecasts are meaningful at that precision?

[peterdudfield]

Good question to ask about s.f.,

the above example of 444000000 to 444234720 seems to be only correct to 3 s.f. though. (and others above are 4.s.f)

Well PVLive changes its capacity daily. The change seen today is 20233.64 to 20234.11, so I guess here, its 7 s.f., but 6.s.f would probably good enough

Sorry if Im creating more issues that its worth, just trying to help. I suspect 5 would actually be ok, also as the use cases shown above are sorted

[devsjc]

I'll look into why they seem to have a lower accuracy than expected. We should be able to do 20233 -> 20234, so I'll see why that isn't tracking.

Don't worry about the issues; I said to make this one an issue - it definitely is an issue!

[peterdudfield]

More recent changes of national capacity are (for context)

• 20233.169
• 20233.64
• 20234.11
• 20234.582
• 20235.053
• 20235.524

[peterdudfield]

I'll look into why they seem to have a lower accuracy than expected. We should be able to do 20233 -> 20234, so I'll see why that isn't tracking.

Don't worry about the issues; I said to make this one an issue - it definitely is an issue!

yea MW is fine here, so 5 s.f.

you got any unittests on this function?

[peterdudfield]

by 20233 might not be an issue, 1398373 to 139800 did seem to be, I would have thought it goes to 1398400

UPDATE I can see you do have unitt tests for this

[peterdudfield]

I manually added some unit tests case

		{1398373, 1398, 3, false}, // not 5.s.f.
		{444234720, 444, 6, false}, // not 5.s.f

to here which pass, but are not 5.s.f