How to treat packages without an OSI-approved license?

[SimonDanisch]

Had a quick chat with @StefanKarpinski about BonitoBook's dual license (PolyForm + commercial) disqualifying it from General registry.
I get the reasoning behind it, but its kind of sad, that I cant use any of the nice registry features (search, discoverability, compatibility checks, versions) for a package, which I'm totally fine to distribute free of charge, since I only want to charge for heavy usage.
Some notes from our chat:
Registry already ships some non-OSS artifacts, so policy may be worth discussing.
Custom registry for one package creates worse UX than URL installation.

Some Solutions:

• Multiple official registries with prompt on first Pkg use where users select which to enable
• Maybe have an official permissive dual license which doesn't have restrictions for distribution by Pkg
• User prompt when installing packages with non-OSS licenses for first time

I guess some of these ideas are longer out in future, but I was wondering if there's also any short term solution to get BonitoBook easy to install and depend on.

[ViralBShah]

My personal view is that General should only care about whether the package license allows distribution (and not necessarily that it is open source), since it already distributes various non-OSS things like MKL and CUDA.

[DilumAluthge]

Is the question here about binary artifacts, or Julia source code?

For binary artifacts, yes we have the MKL artifacts (MKL_jll) and the CUDA artifacts (CUDA_jll), neither of which are open-source.

For Julia source code, we have historically always required an OSI license. My personal opinion is that we should preserve that requirement for Julia source code (although I can't speak for all General registry maintainers).

[ViralBShah]

So, why should Julia source code be treated differently? For example if I compile a julia code with juliac and make an artifact, would different rules apply?

[DilumAluthge]

For example if I compile a julia code with juliac and make an artifact, would different rules apply?

In that case, if the original Julia source code wasn't available under an OSI-approved license, then in my opinion we shouldn't allow the corresponding binary artifacts in the General registry.

---

In my opinion, there's a couple different reasons for treating Julia source code differently.

1. Pragmatic: For MKL or CUDA, there's no realistic chance that we'd be able to convince Intel or NVIDIA to open-source them, even if we told them we'd remove MKL or CUDA from the General registry. In contrast, for Julia packages, we can actually ask package authors to open-source their packages, because package authors are usually active participants in the Julia community.
2. Philosophical: What is the purpose of the General registry? In my opinion, I think that the purpose of the General registry should be to distribute open-source Julia packages and to foster the growth of the open-source Julia ecosystem. Requiring Julia source code to be open-source helps further this mission.

[DilumAluthge]

but I was wondering if there's also any short term solution to get BonitoBook easy to install and depend on.

Can you give some more details on the specific pain points that you're running into? I do want it to be easy for people to use unregistered packages.

For simple usage, installing by URL is pretty easy right?

] add https://github.com/SimonDanisch/BonitoBook.jl

And for more complicated use cases, you can use the [sources] section of Project.toml, which IIUC is designed exactly for the use case of unregistered packages - and doesn't require distributing Manifest.toml.

Other solutions include:

1. Distributing Manifest.toml files.
2. Custom registry (which may be suitable for complicated commercial use cases).

Are there use cases that you're running into that aren't covered by the above?

Or are there pain points you're hitting with the above solutions? Perhaps there are improvements we can make to the above features?

[DilumAluthge]

Custom registry for one package creates worse UX than URL installation.

Could you say a little more about this? To me, naively it seems that the custom registry workflow is pretty simple (just two lines):

] registry add https://custom-registry-url

] add MyPackageName

To me, that seems pretty similar to the URL installation workflow (] add https://package-url).

[SimonDanisch]

call me naive, but it's literally open source and free to distribute ;) just not with unlimited usage for someone earning lots of money with the software ...
Anyways the whole pipeline of registering a new version is super polished with General, catching mistakes, running tests automatically creating tags with release notes, etc... I guess the main question is, should this service apply to a software which is free to 95% of the Julia community instead of 100%. It would actually be a pleasure to put in some money to finance this amazing infrastructure, if i ever earn substantial money from this, so maybe this could be a way to bolster general? might be a slippery slope though, and hard to do right legally 🤷

The source section is nice, but it's also awkward to work with, im not even sure how it works with version conflicts and resolving? I would expect it not to take part in pkgs compat bound resolution?
Also, I got a complaint that installing from source doesn't work for their company firewall ( not actually sure what's the problem there and why it works once it's registered).

[ViralBShah]

1. Pragmatic: For MKL or CUDA, there's no realistic chance that we'd be able to convince Intel or NVIDIA to open-source them, even if we told them we'd remove MKL or CUDA from the General registry. In contrast, for Julia packages, we can actually ask package authors to open-source their packages, because package authors are usually active participants in the Julia community.

I feel this is actually not pragmatic - penalizing everyone else who is not big is not a good reason. Also, it feels a bit arbitrary, in that how do we decide what classifies as big. How about Gurobi_jll? That is a commercial solver that is already in the General registry, and you couldn't convince them to open source Gurobi either.

2. Philosophical: What is the purpose of the General registry? In my opinion, I think that the purpose of the General registry should be to distribute open-source Julia packages and to foster the growth of the open-source Julia ecosystem. Requiring Julia source code to be open-source helps further this mission.

I would be more onboard with a philosophical mission, but in that case, it would be best to apply it uniformly and remove MKL, CUDA, Gurobi and any others. The users clearly want these, and not allowing these would perhaps even reduce the size of the community.

I believe the purpose of the General registry is to make software available to Julia users in an easy way. In fact one could argue the mission is better served by allowing broader distribution, which would make it possible for more companies and people to choose Julia, which would create a vibrant community of Julia devs.

[DilumAluthge]

[Forgive the delay in response - I'm out of office a few days this week, so I'll write up some more thoughts when I'm back.]

[ViralBShah]

One thing worth considering is if we should do a separate registry like Debian’s non-free.

[SimonDanisch]

Just cross referencing @ericphanson idea of a tag system as a possible solution:
https://discourse.julialang.org/t/should-general-have-a-guideline-or-rule-preventing-registration-of-vibe-coded-packages/133205/52?u=sdanisch

[ericphanson]

I think there is a point in distinguishing between Julia source code and artifacts. Julia is a very compositional and interactive language. You can’t @edit your way into a binary or @less and see some transitive dependency code that isn’t open source. Or @eval a monkey patch.

Sure, even OSI code comes with conditions, but it at least promises you can use, modify, and share the code. With a non-OSI license we may be able to distribute it, but you the user may not be able to use it or modify it.

It feels like a trap! This code is in your session, fully and dynamically composing with your program, but you don’t have your usual rights.

The other way it’s different is just that the actual point of General, its reason to exist, is to distribute Julia code. It just seems fine if there are different rules for artifacts because they are incidental. I know legally it doesn’t really matter though.

Also, I think you’re aware, the Julia source code of Gurobi_jll (that thin shell) is indeed MIT licensed.

—-

Since non-OSI Julia code is currently not permitted though, if we do want to go this path we have the option of making it opt-in. Eg using tags as @SimonDanisch wrote, we could have Pkg only use the OSI-license tagged subset by default and require setting a preference to get non-OSI code.

I guess we might also want a stricter tag for all code and all artifacts are also freely licensed.

[ViralBShah]

Sure, even OSI code comes with conditions, but it at least promises you can use, modify, and share the code. With a non-OSI license we may be able to distribute it, but you the user may not be able to use it or modify it.

By this definition, @SimonDanisch 's code meets the requirements and should be included. The fact that certain users have to pay for it does not take away their other rights (if I have understood the license correctly).

The other way it’s different is just that the actual point of General, its reason to exist, is to distribute Julia code. It just seems fine if there are different rules for artifacts because they are incidental. I know legally it doesn’t really matter though.

While I understand the sentiment - this now means that I am better off either writing my code in a non-Julia langauge and compiling a binary for inclusion in General as an artifact, or perhaps even creating a binary with juliac.

Also, I think you’re aware, the Julia source code of Gurobi_jll (that thin shell) is indeed MIT licensed.

Yes. My point was about the artifact - and to highlight the point that we do distribute non open source software that is not as widely used as MKL or CUDA.

Since non-OSI Julia code is currently not permitted though, if we do want to go this path we have the option of making it opt-in. Eg using tags as @SimonDanisch wrote, we could have Pkg only use the OSI-license tagged subset by default and require setting a preference to get non-OSI code.

I guess we might also want a stricter tag for all code and all artifacts are also freely licensed.

I like these suggestions as they would bring uniformity.

I am personally not advocating a specific solution. My point is mainly about consistency and uniformity, and currently things are a bit all over the place.

[ericphanson]

Yeah, I agree it’s a somewhat arbitrary line. But if you do write your code in another language or with juliac and call into from an open source wrapper, there is at least a clear technical line in your Julia session where the non-OSI code lies (on the other side of the ccalls). I guess with sufficient technical work one could blur this line and make @edit and other introspections dig even into the binary artifact if it’s Julia code, somehow, and merge method tables and stuff like that. And that would kind of defeat the separation. But it seems like it currently is a real separation.

[ericphanson]

The other issue I have is then philosophy of maintainership. If General is about serving other people’s paid-for software it feels weird to volunteer to do that work for free. It feels like eventually we’d have to move to some model like a foundation that hires a couple people and tries to scrape funding from big corporate users etc.

I’m not saying I wouldn’t work on it anymore but I would definitely need to think hard about it at least. I’m not sure how other maintainers feel about it. Maybe it’s not that different from folks using a package I wrote for a corporate use case or something like that. It kind of feels different though. Especially if there’s any change in expectations (“we’re losing money bc tool X is down, go fix it!!”).