envconsul deprecation

[typkrft]

Output of brew config

NA

Output of brew doctor

NA

Description of issue

I didn't want to setup a pull request for a formulae that was intentionally deprecated before I understood why. envconsul comes a precompiled binary or is installed via go. The deprecation message states that it's dependent on deprecated consul. It lets you interact with consul and vault but is not a dependency of.

[gromgit]

Putting together the cryptic clues you left, I assume you're talking about fabio.

I don't use any of the products you mention, but a quick source check reveals that there are at least two "points of contact" that fabio has with Consul:

1. Importing the Consul API source at build time. HashiCorp has publicly committed to keeping all their APIs licensed under MPL-2.0, so this should not be an issue.
2. Testing against, and being configured by, a running Consul. This seems to be the real stumbling block, and not something easily worked around.

[gromgit]

Vault is not an option, since it's also deprecated for the same reason.

[typkrft]

Vault and Consul deprecation is due to a license change, which makes sense. envconsul is not changing licenses that I'm aware of. It seems like poor form to remove open source tools because of a test dependency. There is plenty of opensource software that supports proprietary software. How do those tests work?

A few questions:
Are test dependencies required in core?
Is envconsul --version and a sha enough verification?
Is there a mechanism for grabbing precompiled packages in a brew formulae? If so they release precompiled packages, which I would assume negate the need to test.

Is it strange that brew is testing software against other software and not against itself where possible for expected behavior? Whats the reasoning behind that? Envconsul is it's own program even if its developed to use in conjunction with other software.

[typkrft]

An example might be 'https://github.com/Homebrew/homebrew-core/blob/master/Formula/e/exercism.rb'.

assert_match version.to_s, shell_output("#{bin}/exercism version")

[gromgit]

There is plenty of opensource software that supports proprietary software. How do those tests work?

Depending on the author, it ranges from "manually, and only if someone reports an issue" to a full GitHub-hosted CI environment like Homebrew.

Are test dependencies required in core?

Yes if they're used for the test. Homebrew's CI starts with a clean slate every time, so not declaring dependencies on stuff you use in the test will naturally result in a failed test.

Is envconsul --version and a sha enough verification?

Not under normal circumstances. Exceptions might be granted if you can convince the Homebrew maintainers otherwise, but "help" or "version" tests are generally considered insufficient.

Is there a mechanism for grabbing precompiled packages in a brew formulae?

Sure. Many third-party formulae hosted by software authors have their url point to a binary release tarball, from which they install their desired assets.

But that's not allowed in the core tap.

Is it strange that brew is testing software against other software and not against itself where possible for expected behavior? Whats the reasoning behind that?

Tests are meant to catch issues with proper software functioning, otherwise they might as well not be run. The majority of tests in Homebrew formulae "tests against itself", simply because they don't need anything else to test proper functioning. Other formulae like envconsul and pgcron can't be usefully tested without their "host" software (Consul/Vault and PostgreSQL respectively); those dependencies would have to be installed by CI during the test phase.

assert_match version.to_s, shell_output("#{bin}/exercism version")

I hadn't even started using Homebrew when the exercism formula was added, but since it's a client for a specific online service that can't be reliably replicated in a CI environment,¹ this "test of limited utility" was allowed to stand.

All that said, you might be able to convince the Homebrew maintainers to look the other way and downgrade the test's usefulness (it was clearly done for exercism after all), but that starts with filing a PR with the desired changes and providing justification. Nothing happens until you do that.

Footnotes

1. I also seem to recall plans to limit network access in Homebrew CI to specific phases for security reasons, and that the test phase is not included in that allowlist. ↩

[typkrft]

Genuinely appreciate the answers and insight into the these issues. I don't think this is a hill I want to die on, but this has opened my eyes regarding the decisions maintainers deal with and design of brew. Personally I'm still conflicted with this train of thought, but it's also completely reasonable to just utilize the wonderful tap system that exists.

Thank you!

[p-linnane]

Thank you @gromgit as usual for a well thought out and formatted response 💜

@typkrft If you open a PR with your proposed changes, we'll happily review it. Just speak to your rationale and link back to this discussion for context. No guarantees it will be undeprecated, but we'll at least see if we can make it work.