Re-analysis of ledger apply/reapply on mainnet (#630)

* Script for organizing ledger operations dataset
* Added read-me file
* Linear models for apply, reapply, and apply-reapply
* Script for running `db-analyser`
* Quantile regression
* Added figures
* Wrote findngs

Author: bwbush

docs/post-cip-findings.md

@@ -2,6 +2,12 @@
 
 This document catalogs Leios-related findings and artifacts that were created subsequent to the [Leios CIP](https://github.com/cardano-foundation/CIPs/pull/1078).
 
+1. [Markovian model of Linear Leios](#markovian-model-of-linear-leios)
+2. [Analysis of UTxO set size and UTxO lifetime](#analysis-of-utxo-set-size-and-utxo-lifetime)
+3. [CPU cost of `Apply`, `Reapply`, and Plutus ledger operations](#analysis-of-utxo-set-size-and-utxo-lifetime)
+
+---
+
 ## Markovian model of Linear Leios
 
 [Markovian simulation of Linear Leios](../analysis/markov/) computes the probability of EB certifications as RBs are produced.
@@ -35,6 +41,8 @@ The figure below shows example results for the probability distribution of the n
 
 ![Example results](../analysis/markov/example-results.png)
 
+---
+
 ## Analysis of UTxO set size and UTxO lifetime
 
 [Analysis of Cardano mainnet](../post-cip/tx-lifetime/tx-lifetime.ipynb) indicates that the number of active UTxOs has leveled off at approximately 11 million unspent transaction outputs. The data likely is not sufficient to build a statistical model to forecast the size of the UTxO set as a function of demand: a more speculative model would be needed.
@@ -50,3 +58,31 @@ The left plot is on a square-root scale horizontally, so one can see how big the
 |                                                                                                                 |                                                                                                                  |
 | --------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
 | ![Distribution of UTxO lifetime on mainnet (square-root scale)](../post-cip/tx-lifetime/utxo-lifetime-sqrt.svg) | ![Distribution of UTxO lifetime on mainnet (logarithmic scale)](../post-cip/tx-lifetime/utxo-lifetime-log10.svg) |
+
+---
+
+## CPU cost of `Apply`, `Reapply`, and Plutus ledger operations
+
+The Jupyter notebook [post-cip/apply-reapply/analysis.ipynb](../post-cip/apply-reapply/analysis.ipynb) analyzes `db-analyser` measurements for Cardano `mainnet`. Linear models and quantile regressions were applied to the dataset in order to estimate how CPU resources scale with block size, transaction count, number of transaction inputs, and number of Plutus steps. These results can be used for reasoning about feasible values of Leios protocol parameters.
+
+Regarding Plutus, nominally, one step unit corresponds to one picosecond on the benchmark machine and one memory unit corresponds to eight bytes allocated on that machine. The following plots show the relationship between execution steps and CPU on the machine where the `db-analyser` experiment was conducted.
+
+| Plutus steps vs CPU usage                                                                 | CPU usage per Plutus step                                                               |
+| ----------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
+| ![CPU usage vs Plutus steps](../post-cip/apply-reapply/steps-picoseconds-scatterplot.png) | ![CPU usage per Plutus step](../post-cip/apply-reapply/steps-picoseconds-histogram.png) |
+
+The db-analyser is very noisy and hard to fit. Nevertheless, here are the best fits obtained using linear models and the quantile regression. Note that the quantile regression was based on a random subset of the data because it is not computationally feasible to perform quantile regression on such a large dataset in a reasonable amount of time.
+
+| Regression      | Dependent variable |   Block size | Number of transactions | Number of transaction inputs | Number of Plutus steps |
+| --------------- | ------------------ | -----------: | ---------------------: | ---------------------------: | ---------------------: |
+| Simple ratio    | `Apply - Reapply`  |              |                        |                              |        `1.5e0 ps/step` |
+| Linear model    | `Apply - Reapply`  | `4.7e4 ps/B` |          `2.3e8 ps/tx` |              `8.0e3 ps/txin` |       `6.1e-1 ps/step` |
+|                 | `Reapply`          | `2.8e3 ps/B` |          `3.5e7 ps/tx` |              `5.2e6 ps/txin` |                        |
+|                 | `Apply`            | `4.8e4 ps/B` |          `1.6e8 ps/tx` |              `1.3e7 ps/txin` |       `6.3e-1 ps/step` |
+| 75th percentile | `Apply - Reapply`  | `1.6e4 ps/B` |          `1.7e8 ps/tx` |              `1.8e7 ps/txin` |       `9.6e-1 ps/step` |
+|                 | `Reapply`          | `1.7e3 ps/B` |          `4.2e7 ps/tx` |              `5.7e6 ps/txin` |                        |
+|                 | `Apply`            | `1.6e4 ps/B` |          `2.2e8 ps/tx` |              `2.3e7 ps/txin` |       `9.7e-1 ps/step` |
+
+Coarsely, the "one picosecond per Plutus step" is a reasonable estimate for Plutus costs; we did not assess whether "eight bytes per Plutus memory unit" was also reasonable.
+
+---

post-cip/apply-reapply/.gitignore

@@ -0,0 +1,2 @@
+ledger-ops-65836843.csv.gz
+*.png

post-cip/apply-reapply/ReadMe.md

@@ -0,0 +1,9 @@
+# Re-analysis of ledger apply/reapply operations on mainnet
+
+Deepened analysis of the `db-analyser` output on the `Apply` and `Reapply` operations for the Cardano mainnet ledger.
+
+
+## Artifacts
+
+- [ledger-ops.sql](ledger-ops.sql): SQL script for merging measurements with ledger information.
+- [analysis.ipynb](analysis.ipynb): Jupyter R notebook for analyzing data and plotting results.