Skip to content

Conversation

@igor-sirotin
Copy link
Collaborator

@igor-sirotin igor-sirotin commented Sep 24, 2025

Requires:

Description

Stop vendoring Go dependencies. Delete vendor/ directory.

Reasoning

As for today, vendoring dependencies in Go is more of a burden than real need.
Major Go projects like go-ethereum don't vendor.

The only reason for vendoring is to have all dependencies on hands in case one of them is suddenly becomes unavailable
(deleted of removed from public). On the other hand, it makes the repo heavier and PRs less clear, which we believe is more important. This becomes more obvious with the usage of go tool (e.g. in PRs like #6933).

nix

With this change, developers will need to update this vendorHash manually:

vendorHash = "sha256-elLrG5/mjsZy8Y2pZ1NnBvbXvVc7d2uvd8oHIoe5H7U=";

I implemented make vendor target for this:

status-go/Makefile

Lines 404 to 412 in 33218c6

# Temporarily redirect the well-known `vendor` target to `vendor-hash`.
# This target will be removed in the future.
vendor: vendor-hash
# https://gitlab.com/peerdb/peerdb/-/blob/d1dbd0c6533dca16bf57322b57cc8fb6ab897a66/nix-update.sh
# After stopping vendoring https://github.com/status-im/status-go/pull/6951,
# we have to manually update `vendorHash` in the nix derivation.
# This target runs the nix build, extracts the expected vendorHash and updates with it the nix derivation.
vendor-hash:

@status-im-auto
Copy link
Member

status-im-auto commented Sep 24, 2025

Jenkins Builds

Click to see older builds (120)
Commit #️⃣ Finished (UTC) Duration Platform Result
772aa31 #1 2025-09-24 18:54:23 ~1 min linux/nwaku 📄log
772aa31 #1 2025-09-24 18:54:37 ~1 min windows/status-go 📄log
✔️ 772aa31 #1 2025-09-24 18:56:36 ~3 min linux/status-go 📦zip
✔️ 772aa31 #1 2025-09-24 18:59:38 ~6 min macos/status-go 📦zip
✔️ 772aa31 #1 2025-09-24 19:04:39 ~11 min tests-rpc 📄log
✔️ 772aa31 #1 2025-09-24 19:19:06 ~25 min tests 📄log
bcc7833 #2 2025-09-25 08:54:24 ~1 min linux/nwaku 📄log
✔️ bcc7833 #2 2025-09-25 08:55:44 ~3 min linux/status-go 📦zip
✔️ bcc7833 #2 2025-09-25 08:56:14 ~3 min macos/status-go 📦zip
✔️ bcc7833 #2 2025-09-25 08:58:20 ~5 min windows/status-go 📦zip
✔️ bcc7833 #2 2025-09-25 09:04:17 ~11 min tests-rpc 📄log
✔️ bcc7833 #2 2025-09-25 09:18:21 ~25 min tests 📄log
✔️ 485d724 #3 2025-09-26 16:06:33 ~3 min linux/status-go 📦zip
✔️ 485d724 #3 2025-09-26 16:06:57 ~3 min macos/status-go 📦zip
✔️ 485d724 #3 2025-09-26 16:09:30 ~5 min windows/status-go 📦zip
✔️ 485d724 #4 2025-09-26 16:10:43 ~3 min macos/status-go 📦zip
✔️ 485d724 #4 2025-09-26 16:11:04 ~4 min linux/status-go 📦zip
✔️ 485d724 #3 2025-09-26 16:13:52 ~10 min linux/nwaku 📦zip
✔️ 485d724 #3 2025-09-26 16:16:58 ~13 min tests-rpc 📄log
✔️ 485d724 #4 2025-09-26 16:24:44 ~10 min linux/nwaku 📦zip
✖️ 485d724 #4 2025-09-26 16:28:00 ~10 min tests-rpc 📄log
✔️ 485d724 #3 2025-09-26 16:31:33 ~27 min tests 📄log
✔️ 485d724 #5 2025-09-26 16:40:21 ~8 min tests-rpc 📄log
✔️ b288939 #5 2025-09-28 18:36:14 ~3 min linux/status-go 📦zip
✔️ b288939 #5 2025-09-28 18:36:29 ~3 min macos/status-go 📦zip
✔️ b288939 #4 2025-09-28 18:39:54 ~7 min windows/status-go 📦zip
✔️ b288939 #6 2025-09-28 18:41:57 ~9 min tests-rpc 📄log
✔️ b288939 #5 2025-09-28 18:43:05 ~10 min linux/nwaku 📦zip
✔️ b288939 #4 2025-09-28 18:58:38 ~25 min tests 📄log
✔️ e59e021 #6 2025-09-28 19:13:28 ~3 min macos/status-go 📦zip
✔️ e59e021 #6 2025-09-28 19:13:47 ~4 min linux/status-go 📦zip
✔️ e59e021 #5 2025-09-28 19:16:27 ~6 min windows/status-go 📦zip
✔️ e59e021 #7 2025-09-28 19:20:53 ~11 min tests-rpc 📄log
✔️ e59e021 #6 2025-09-28 19:21:09 ~11 min linux/nwaku 📦zip
✔️ e59e021 #5 2025-09-28 19:37:29 ~27 min tests 📄log
✔️ dde9a6b #7 2025-09-28 19:55:31 ~3 min linux/status-go 📦zip
✔️ dde9a6b #7 2025-09-28 19:55:46 ~3 min macos/status-go 📦zip
✔️ dde9a6b #6 2025-09-28 19:58:41 ~6 min windows/status-go 📦zip
✔️ dde9a6b #8 2025-09-28 20:01:06 ~9 min tests-rpc 📄log
✔️ dde9a6b #7 2025-09-28 20:02:22 ~10 min linux/nwaku 📦zip
✔️ dde9a6b #6 2025-09-28 20:18:06 ~26 min tests 📄log
✔️ dde9a6b #8 2025-09-29 21:46:59 ~3 min macos/status-go 📦zip
✔️ dde9a6b #8 2025-09-29 21:47:20 ~3 min linux/status-go 📦zip
✔️ dde9a6b #7 2025-09-29 21:50:08 ~6 min windows/status-go 📦zip
✖️ dde9a6b #9 2025-09-29 21:53:24 ~9 min tests-rpc 📄log
✔️ dde9a6b #8 2025-09-29 21:54:22 ~11 min linux/nwaku 📦zip
✖️ dde9a6b #10 2025-09-29 22:07:10 ~7 min tests-rpc 📄log
✔️ dde9a6b #7 2025-09-29 22:09:44 ~26 min tests 📄log
✖️ c61255e #8 2025-09-30 13:01:21 ~2 min tests 📄log
c61255e #9 2025-09-30 13:01:54 ~3 min linux/nwaku 📄log
✔️ c61255e #9 2025-09-30 13:02:16 ~3 min linux/status-go 📦zip
✔️ c61255e #9 2025-09-30 13:02:34 ~3 min macos/status-go 📦zip
✔️ c61255e #8 2025-09-30 13:03:55 ~4 min windows/status-go 📦zip
✖️ c61255e #11 2025-09-30 13:08:31 ~9 min tests-rpc 📄log
b4cc062 #10 2025-09-30 13:27:27 ~3 min linux/nwaku 📄log
✔️ b4cc062 #10 2025-09-30 13:27:42 ~3 min linux/status-go 📦zip
✔️ b4cc062 #10 2025-09-30 13:28:07 ~3 min macos/status-go 📦zip
✔️ b4cc062 #9 2025-09-30 13:30:22 ~6 min windows/status-go 📦zip
✔️ b4cc062 #12 2025-09-30 13:33:44 ~9 min tests-rpc 📄log
✔️ b4cc062 #9 2025-09-30 13:50:24 ~26 min tests 📄log
daa5f1a #11 2025-10-28 22:41:44 ~1 min linux/nwaku 📄log
daa5f1a #12 2025-10-28 22:43:16 ~1 min linux/nwaku 📄log
✔️ daa5f1a #11 2025-10-28 22:44:03 ~3 min macos/status-go 📦zip
✔️ daa5f1a #11 2025-10-28 22:44:35 ~4 min linux/status-go 📦zip
✖️ daa5f1a #10 2025-10-28 22:45:43 ~5 min tests 📄log
✔️ daa5f1a #10 2025-10-28 22:47:09 ~6 min windows/status-go 📦zip
✔️ daa5f1a #12 2025-10-28 22:47:59 ~3 min macos/status-go 📦zip
✔️ daa5f1a #12 2025-10-28 22:49:01 ~4 min linux/status-go 📦zip
✖️ daa5f1a #13 2025-10-28 23:01:55 ~21 min tests-rpc 📄log
✔️ 92c8018 #13 2025-10-28 22:54:05 ~3 min linux/status-go 📦zip
✔️ 92c8018 #13 2025-10-28 22:54:37 ~4 min macos/status-go 📦zip
✔️ 92c8018 #11 2025-10-28 22:56:06 ~5 min windows/status-go 📦zip
✔️ 92c8018 #13 2025-10-28 23:13:51 ~23 min linux/nwaku 📦zip
✔️ 92c8018 #11 2025-10-28 23:18:25 ~27 min tests 📄log
✔️ f824870 #14 2025-10-28 23:05:38 ~3 min linux/status-go 📦zip
✔️ f824870 #14 2025-10-28 23:05:44 ~3 min macos/status-go 📦zip
✔️ f824870 #12 2025-10-28 23:07:37 ~5 min windows/status-go 📦zip
✔️ f824870 #14 2025-10-28 23:15:40 ~13 min tests-rpc 📄log
✔️ f824870 #14 2025-10-28 23:36:40 ~22 min linux/nwaku 📦zip
✔️ f824870 #12 2025-10-28 23:45:29 ~26 min tests 📄log
✔️ f824870 #13 2025-10-30 11:52:59 ~4 min windows/status-go 📦zip
✔️ f824870 #15 2025-10-30 11:55:02 ~7 min linux/status-go 📦zip
✔️ f824870 #15 2025-10-30 11:55:04 ~7 min macos/status-go 📦zip
✔️ f824870 #15 2025-10-30 12:15:48 ~27 min linux/nwaku 📦zip
✔️ d1d2aec #14 2025-10-30 11:57:52 ~4 min windows/status-go 📦zip
✔️ d1d2aec #16 2025-10-30 12:19:17 ~24 min macos/status-go 📦zip
✔️ d1d2aec #16 2025-10-30 12:19:33 ~24 min linux/status-go 📦zip
✔️ d1d2aec #16 2025-10-30 12:29:47 ~34 min tests-rpc 📄log
✔️ d1d2aec #14 2025-10-30 12:43:28 ~47 min tests 📄log
✔️ d1d2aec #16 2025-10-30 12:46:31 ~30 min linux/nwaku 📦zip
✔️ ae5ed83 #17 2025-11-01 13:25:04 ~3 min macos/status-go 📦zip
✔️ ae5ed83 #17 2025-11-01 13:25:27 ~4 min linux/status-go 📦zip
✔️ ae5ed83 #15 2025-11-01 13:27:11 ~5 min windows/status-go 📦zip
✔️ ae5ed83 #17 2025-11-01 13:31:49 ~10 min linux/nwaku 📦zip
✔️ ae5ed83 #17 2025-11-01 13:34:53 ~13 min tests-rpc 📄log
✔️ ae5ed83 #15 2025-11-01 13:48:16 ~26 min tests 📄log
✔️ c9a2aac #18 2025-11-02 11:52:29 ~3 min linux/status-go 📦zip
✔️ c9a2aac #16 2025-11-02 11:53:43 ~4 min windows/status-go 📦zip
✔️ c9a2aac #18 2025-11-02 11:55:09 ~6 min macos/status-go 📦zip
✔️ c9a2aac #18 2025-11-02 12:00:23 ~11 min linux/nwaku 📦zip
✔️ c9a2aac #18 2025-11-02 12:01:24 ~12 min tests-rpc 📄log
✔️ c9a2aac #16 2025-11-02 12:16:42 ~27 min tests 📄log
✔️ d06b803 #19 2025-11-02 11:58:51 ~3 min linux/status-go 📦zip
✔️ d06b803 #19 2025-11-02 12:01:17 ~5 min macos/status-go 📦zip
✔️ d06b803 #17 2025-11-02 12:01:35 ~6 min windows/status-go 📦zip
✔️ d06b803 #20 2025-11-02 12:02:33 ~3 min linux/status-go 📦zip
✔️ d06b803 #19 2025-11-02 12:11:24 ~10 min linux/nwaku 📦zip
✔️ d06b803 #19 2025-11-02 12:13:56 ~12 min tests-rpc 📄log
✔️ ab382cb #21 2025-11-02 12:07:30 ~3 min linux/status-go 📦zip
✔️ ab382cb #20 2025-11-02 12:09:41 ~5 min macos/status-go 📦zip
✔️ ab382cb #18 2025-11-02 12:09:50 ~5 min windows/status-go 📦zip
✔️ ab382cb #20 2025-11-02 12:23:12 ~11 min linux/nwaku 📦zip
✔️ ab382cb #20 2025-11-02 12:27:11 ~13 min tests-rpc 📄log
✔️ ab382cb #17 2025-11-02 12:43:25 ~26 min tests 📄log
✔️ ab382cb #22 2025-11-05 16:35:11 ~3 min linux/status-go 📦zip
✔️ ab382cb #21 2025-11-05 16:35:28 ~3 min macos/status-go 📦zip
✔️ ab382cb #19 2025-11-05 16:37:48 ~5 min windows/status-go 📦zip
✔️ ab382cb #21 2025-11-05 16:42:28 ~10 min linux/nwaku 📦zip
✔️ ab382cb #21 2025-11-05 16:44:56 ~13 min tests-rpc 📄log
✔️ ab382cb #18 2025-11-05 16:59:15 ~27 min tests 📄log
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ e08b53b #23 2025-11-05 16:38:44 ~3 min linux/status-go 📦zip
✔️ e08b53b #22 2025-11-05 16:39:25 ~3 min macos/status-go 📦zip
✔️ e08b53b #20 2025-11-05 16:43:42 ~5 min windows/status-go 📦zip
✔️ e08b53b #22 2025-11-05 16:54:17 ~11 min linux/nwaku 📦zip
✔️ e08b53b #22 2025-11-05 16:58:26 ~13 min tests-rpc 📄log
✔️ a097641 #24 2025-11-05 16:54:03 ~3 min linux/status-go 📦zip
✔️ a097641 #23 2025-11-05 16:54:16 ~3 min macos/status-go 📦zip
✔️ a097641 #21 2025-11-05 16:56:15 ~5 min windows/status-go 📦zip
✔️ a097641 #23 2025-11-05 17:06:50 ~12 min linux/nwaku 📦zip
✖️ a097641 #23 2025-11-05 17:11:18 ~12 min tests-rpc 📄log
✔️ a097641 #24 2025-11-05 17:26:02 ~11 min tests-rpc 📄log
✔️ a097641 #19 2025-11-05 17:26:38 ~27 min tests 📄log

@codecov
Copy link

codecov bot commented Sep 25, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.63%. Comparing base (0191cb4) to head (a097641).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #6951      +/-   ##
===========================================
+ Coverage    59.58%   59.63%   +0.04%     
===========================================
  Files          801      801              
  Lines       113885   113885              
===========================================
+ Hits         67864    67914      +50     
+ Misses       39085    39053      -32     
+ Partials      6936     6918      -18     
Flag Coverage Δ
functional 34.80% <ø> (+0.22%) ⬆️
unit 55.42% <ø> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 37 files with indirect coverage changes

@igor-sirotin
Copy link
Collaborator Author

@status-im/devops guys can you please assist me with -nix builds?
I tried to understand what's going on there, but couldn't get it 🤔

@igor-sirotin igor-sirotin changed the base branch from chore/nwaku-submodule to feat/bump-waku-go-bindings September 29, 2025 21:43
@igor-sirotin igor-sirotin moved this from In Progress to Blocked in Status Desktop/Mobile Board Sep 30, 2025
@jakubgs
Copy link
Member

jakubgs commented Sep 30, 2025

The trick is actually that vendorHash cannot be null if there's no vendor folder:

      # The SRI hash of the vendored dependencies.
      # If `vendorHash` is `null`, no dependencies are fetched and
      # the build relies on the vendor folder within the source.
      vendorHash ? throw (
        if args ? vendorSha256 then
          "buildGoModule: Expect vendorHash instead of vendorSha256"
        else
          "buildGoModule: vendorHash is missing"
      ),

https://github.com/NixOS/nixpkgs/blob/7ff837017c3b82bd3671932599a119d7bc672ff0/pkgs/build-support/go/module.nix#L30-L38

So from now on we need to specify a real hash. Which means it will have to be updated every time dependencies change.

@igor-sirotin
Copy link
Collaborator Author

The trick is actually that vendorHash cannot be null if there's no vendor folder:

Thanks for fixing this.
I was looking into this as well, but couldn't figure it out.

So from now on we need to specify a real hash. Which means it will have to be updated every time dependencies change.

@jakubgs is there any chance we can avoid this? It's gonna be quite an annoying procedure 😄
Perhaps smth like lib.fakeHash, or smth similar possible?

@jakubgs
Copy link
Member

jakubgs commented Sep 30, 2025

I know of no built-in way of doing this. The purpose of fakeHash is to just be a placeholder for when you have no value yet.
https://dmarcoux.com/posts/hash-in-nix-packages/

We'd have to write some kind of script to generate the hash for us, but it would have to be committed anyway by the devs.

@igor-sirotin igor-sirotin changed the base branch from feat/bump-waku-go-bindings to feat/external-nwaku October 28, 2025 22:40
Base automatically changed from feat/external-nwaku to develop October 30, 2025 11:47
@igor-sirotin igor-sirotin force-pushed the chore/stop-vendoring branch 2 times, most recently from c9a2aac to d06b803 Compare November 2, 2025 11:54
@igor-sirotin igor-sirotin changed the base branch from develop to feat/external-goroutine-defer-guard November 2, 2025 11:54
@igor-sirotin
Copy link
Collaborator Author

lint-panics got flaky with this change. Stable locally, but was most of the times failing on CI.
So I changed its implementation:

@igor-sirotin igor-sirotin marked this pull request as ready for review November 2, 2025 12:27
@igor-sirotin igor-sirotin requested review from a team and 0xM3R as code owners November 2, 2025 12:27
@igor-sirotin igor-sirotin moved this from Blocked to Code Review in Status Desktop/Mobile Board Nov 2, 2025
Copy link
Contributor

@osmaczko osmaczko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🫶

@igor-sirotin igor-sirotin force-pushed the feat/external-goroutine-defer-guard branch from eb76550 to d0edf2d Compare November 5, 2025 15:09
Base automatically changed from feat/external-goroutine-defer-guard to develop November 5, 2025 16:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Code Review

Development

Successfully merging this pull request may close these issues.

4 participants