by gravypod on 1/14/2022, 9:21:15 PM
by cfors on 1/14/2022, 9:43:15 PM
In today's world of excellent CLI tools I don't think grep is a good choice, especially for checking irregular languages like XML. [0]
I use tools like `jq` [1] or `yq` [2] all the time for CI checks. One useful check, is we have a configuration file stored as several hundred lines of YAML. Its a nice thing to maintain a sorted order for that, so we have a git pre-commit hook that runs the following:
> yq eval --inplace '.my_key|= sort' my_file.yaml
Of course, a pre-commit hook or CI both work. There's pros and cons of both. For our team, the pre-commit hook is a low enough level of effort, and doesn't require a CI check for something that executes in milliseconds.
[0] https://stackoverflow.com/a/1732454
by zX41ZdbW on 1/14/2022, 10:18:00 PM
Here is my favorite: https://github.com/ClickHouse/ClickHouse/blob/master/utils/c...
"Too many exclamation marks"
by excircul on 1/14/2022, 9:41:06 PM
> I personally prefer ripgrep as it is much faster than grep, but usually that is not available on CI machines.
I recommend git grep, which is comparable in speed to ripgrep, since it ignores non-tracked files and searches the object storage directly. It is also able to run in parallel.
by eminence32 on 1/14/2022, 9:26:14 PM
The semgrep[1] tool seems like the logical next step, for when you've outgrown plain ol' grep.
by david_allison on 1/14/2022, 9:40:17 PM
Android's default linting already contains a "missing translation" lint rule which you can activate: "MissingTranslation"[0]
For Android specifically: Gradle and Android Studio both support a powerful linting framework (to the level that it can provide auto-fixes to the IDE). It's better to provide an in-editor to guide your contributors before it hits CI, then have CI nag if they didn't fix the in-editor warnings/errors:
Some examples of custom lint rules[1] and the default rules which Android Studio runs[2]:
[0] https://android.googlesource.com/platform/tools/base/+/32923...
[1] https://github.com/ankidroid/Anki-Android/tree/master/lint-r...
[2] https://github.com/ankidroid/Anki-Android/blob/master/lint-r...
by jrockway on 1/14/2022, 10:03:47 PM
Don't you worry about what happens when the translation software produces a semantically equivalent empty string, or something? Like `<string><![CDATA[]]></string>`. An XML parser will find that to be empty, grep will be like "ooh lots of text in there".
by chubot on 1/15/2022, 8:07:07 AM
If you think of a CI configuration as a shell script, then this is normal and not surprising.
A CI config is just a big job that invokes a bunch of tools like "go build" or "npm test", which is exactly what a shell script is too. I would get rid of the YAML and use shell for the whole thing :)
Shell does have some problems, like needing to be more parallel, incremental, and reproducible, but the YAML-based CI systems all have those problems too.
Related: http://www.oilshell.org/blog/2021/04/build-ci-comments.html#... (and the entire post)
by xorcist on 1/14/2022, 11:37:15 PM
Wouldn't quick sanity checks like these make more sense in a git push hook?
No reason to wait for the CI. There's also the risk of broken intermediary commits unless the CI check each and every commit in isolation.
by joatmon-snoo on 1/14/2022, 10:50:20 PM
Shameless plug: if you find yourself wanting more advamced custom painting, you can try http://trunk.io, which provides you with three different ways to write your own linters (pass/fail based on your script's exit code, spit out a patch that should be applied to your code, or LSP diagnostics) that can get propagated to VSCode and CI (as well as just as a CLI tool, if that's your preference).
Disclaimer: I work on trunk :)
by ilyash on 1/15/2022, 6:44:33 PM
Interesting to see how nobody cares about exit code 2. Modify your grep command to have syntax error and you will always think you have all the translations.
Edit: ah.. forgot the whole point.. In Next Generation Shell (author here) this doesn't happen: ngs -e '$(grep ...).not()' does right thing. grep exit code 0 becomes 1, 1 becomes 0, 2 becomes 240 (exception)
by rtuin on 1/14/2022, 9:09:49 PM
Oh the power of Linux.
I’m also using grep in CI as a simple but effectieve check on dependency violations in some codebases.
by jeffbee on 1/14/2022, 9:35:12 PM
grep and other shell tool one-liners also make excellent kubernetes liveness checks. I have some kubernetes daemonsets that don't do anything other than assert that things on the node are as they should, via grep exit status.
by matmann2001 on 1/14/2022, 10:56:22 PM
Wouldn't this example do the incorrect thing in the event of a grep error (exit code 2)?
by indigodaddy on 1/15/2022, 12:26:56 AM
Learn something new everyday.. Never knew about —exclude/include, very cool.
I think this is a great example of where build systems, that understand the need for testing, can help out. In a build system I use quite often (Bazel) you can express this as an `sh_test()` [0] which provides documentation about what you're attempting to do and provides you with a way to reproduce the failure locally. You don't have to push the code and wait for CI to fail to find out, or debug, this error.
Extra fun thing to do: print a message describing why the decision was made and how to resolve the error in the failure message of your test!
[0] - https://docs.bazel.build/versions/main/be/shell.html#sh_test