1
0
mirror of https://github.com/stedolan/jq.git synced 2024-05-11 05:55:39 +00:00

1651 Commits

Author SHA1 Message Date
80bc768c5d jq_fuzz_fixed: fix test cases that use string interpolation
"\(" is implementation-defined in C++, and both clang and gcc treat it
as equivalent to "(", not "\\(".
This patch replaces "\(" with "\\(" so that tests with string
interpolation actually use string interpolation.
2023-12-10 15:22:03 +01:00
5029328d35 tests: add new fuzzer
Signed-off-by: David Korczynski <david@adalogics.com>
2023-12-07 12:57:54 +01:00
20a403e97e jq_fuzz_execute.cpp: fix false assert
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64632 by following the suggestion here: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64632#c2 -- the issue is fixed when this attribute is set.

I did not add any of the other suggestions in the comment: we haven't run into these issues in the fuzzer so far.
2023-12-03 17:54:25 +01:00
693a7261e5 jq_fuzz_execute: fix memory leak
Signed-off-by: David Korczynski <david@adalogics.com>
2023-12-03 03:53:49 +01:00
bfb7fd570f jq_fuzz_execute: cleanup un-needed extern
Signed-off-by: David Korczynski <david@adalogics.com>
2023-11-30 14:40:36 +01:00
252ab244ce Add fuzzer targeting jq_next
Signed-off-by: David Korczynski <david@adalogics.com>
2023-11-30 14:40:36 +01:00
13353515bd jq_fuzz_compile: dump disassembly
Signed-off-by: David Korczynski <david@adalogics.com>
2023-11-30 14:40:36 +01:00
98a206964d Convert decnum to binary64 (double) instead of decimal64
This is what the JSON spec suggests and will also be less confusing compared to other jq implementations and langauges.

Related to #2939
2023-11-29 09:36:33 +01:00
1617091033 website: use https URLs instead of http URLs in download page
Also add markdown formatting for decNumber URL so it gets rendered as a
link in the html page.
2023-11-29 09:35:36 +01:00
d14393f552 Keep releasing executables with legacy names 2023-11-28 22:51:20 +01:00
6b82dd3c47 libjq: extern C for C++
If using libjq from C++ it would be nice to not need to do this at the
import site, so just extern "C" to the public headers for libjq

Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
2023-11-28 22:50:25 +01:00
460a5c12b4 jq_fuzz_parse_extended.c: don't jv_free() twice
jv_dump() frees its argument.

I missed this problem before merging #2952, whoops! =)

fixup from eb3b5654bbd285fa70bab8ca71f2284354adf625
2023-11-28 20:51:07 +01:00
4570c23720 jv_parse: refactor jv_parse_sized
Signed-off-by: David Korczynski <david@adalogics.com>
2023-11-28 20:36:59 +01:00
eb3b5654bb test: add jv_dump to extended fuzzer
Signed-off-by: David Korczynski <david@adalogics.com>
2023-11-28 20:36:59 +01:00
44300e4310 Extend fuzzing set up
Adds a parse function ins `jv_parse.c` that enables parsing using custom
flags for the parser. This is then used by two fuzzers added as well.

This is to make sure fuzzing hits various code parts currently not
fuzzed, e.g. `stream_token`:
https://storage.googleapis.com/oss-fuzz-coverage/jq/reports/20231125/linux/src/jq/src/jv_parse.c.html#L241

Signed-off-by: David Korczynski <david@adalogics.com>
2023-11-28 20:36:59 +01:00
88f01a741c simplify paths/0 and paths/1
`recurse/0` already handles traversing objects and arrays, so it is more
consistent to use that.
For `paths/1` it is easier to use the actual value returned by
`recurse` instead of querying that value with `getpath/1` afterwards.
2023-11-16 21:14:58 +01:00
6c035133e8 README.md: tosed => to sed (#2944) 2023-11-06 23:59:22 +09:00
cca1f7d18f Comment bug fixes, and fully support Tcl-style multiline comments
* bugfix: comments were incorrectly being terminated by CR; for example
    jq -n $'1 #foo\r'
  fails to compile because the CR character terminates the comment, and
  CR is not a valid character in jq syntax.

* improvement: comments fully support Tcl-style line continuation.
  Previously this was only "supported" in `-f' scripts, whose first line
  starts with "#!", and second line starts with # and ends with \, only
  for the comment on the second line, only for one extra line.

* man: document comment syntax, which was previously undocumented.

* tests: add regression tests for the bugfix, and some tests for line
  continuation in comments.
2023-11-03 15:52:02 -05:00
f4929f3c19 src/builtin.c: remove unnecessary jv_copy-s in type_error/type_error2
Also fix doubled semicolon   jv ret;;   =>   jv ret;
2023-10-30 22:12:27 -05:00
d5590f90da jvp_object_contains: remove unnecessary jv_copy 2023-10-30 22:12:27 -05:00
e85e358233 Fix possible uninitialised value dereference if jq_init() fails
If jq_init() fails, goto out would try to free input_state which is
uninitialised. I initialised input_state to NULL to fix the problem.

Ref: https://github.com/jqlang/jq/pull/2934#discussion_r1367795641

Reported-By: Klemens Nanni <kn@openbsd.org>
2023-10-22 10:51:24 -05:00
7ab117a483 Defer heap variable initialisation after pledge
Otherwise `AGRS` and `program_arguments` remain allocated/unfreed in the
early (extremely unlikely) pledge(2) failure case.

Move their allocation before jq_init(), the first case of jumping to
`out` where they are cleaned up, where it also seems to logically fit
better than above between function entry, locale setup and OpenBSD
specific pledge.
2023-10-21 23:30:36 -05:00
77dcaf3fdc Remove unused mkstemp()
b82c231 "Remove -i option (#704)" removed its last usage in 2015.

Spotted while looking for code could potentially write/create/modify files.
2023-10-21 23:30:36 -05:00
f1bfd0c518 Restrict systems operations on OpenBSD
Use pledge(2)[0] to limit jq(1) to reading files.
It does not change files and only writes to standard output/error.
It never deals with TTY, network, process management or other subsystems.

This is to reduce jq's attack surface and potential damage.

OpenBSD is carrying a local patch[1] in its official jq port/package
since 2016.  An improved version:

- drop no longer needed "getpw" promise
  f1c4947 "Avoid getpwuid for static linking" removed getpwuid(3) usage
- pledge before jq_init() to simplify the error path
- use perror(3) to print errno(2)

No behaviour change in tests or real world usage observed on
OpenBSD/amd64 7.4.

0: https://man.openbsd.org/pledge.2
1: https://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/textproc/jq/patches/patch-main_c
2023-10-21 23:30:36 -05:00
9de0e26ce6 Generate links in man page
Some [foo](bar) links were added to manual.yml without updating
build_manpage.py to make it generate roff for "a" tags.

Fixes #2930
2023-10-12 15:32:23 -05:00
c10cbbff34 Remove undefined behavior caught by LLVM 10 UBSAN. 2023-10-08 12:32:22 -05:00
6b6feaffbb Revert "Revert "od -c => od -tc: od -c is an XSI extension..."
This reverts commit 0e70f7a57e08b6229c41ab98d1d9a9bca46625be.

There is no reason to revert this change.

In #2922, I only disagreed with the commit message suggesting that
  LC_CTYPE=C od -t c    is   equivalent to   od -c

The only documented differences are that -tc is required to be
influenced by -N and -j, while -c is not, and that -c is required to
only support a subset of the backslash sequences that -tc should
support.
2023-10-05 15:37:44 +02:00
0e70f7a57e Revert "od -c => od -tc: od -c is an XSI extension equivalent to LC_CTYPE=C od -tc and not universally available"
This reverts commit 0bce9fb8ed0fbaeba0901ff9778756e4e037cd47.
2023-10-04 22:28:24 -05:00
0bce9fb8ed od -c => od -tc: od -c is an XSI extension equivalent to LC_CTYPE=C od -tc and not universally available 2023-10-04 17:46:13 -05:00
4ebd21e1eb Allow passing the inline jq script before --
jq previously only allowed passing the inline script before -- (as if
they were options) even though one would expect the inline script to be
a positional argument.

Since jq previously also refused to run with a usage error if the script
was passed after -- (It was not assuming  .  as script as it does when
no arguments are passed), and positional arguments are allowed before --
and even before other options, it should not be a breaking change to
change that weird behaviour, and allow the script to appear after --.

It also simplifies the option parsing code a bunch.

Fixes #2918
2023-10-04 00:00:46 -05:00
7f547827e4 Simplify pick example
Old pick example included input array in command line, making `input` confusing
and redundant.
2023-10-03 22:48:12 +02:00
6b5a18f036 Actually use number correctly casted from double to int as index
The code was using (int)jv_number_value(k) instead of (int)didx.

follow-up from 0e067ef93605493060392f0999be27694146fca4

Useless assignments to didx detected by clang-tidy.
2023-09-27 22:54:23 -05:00
8206bc8fd2 main.c: Remove unused EXIT_STATUS_EXACT option
In process there is a suspicious  options |= EXIT_STATUS_EXACT  that
is run when the jq script is terminated by halt, or halt_error.

That line of code acutally does nothing because options is a local
argument variable, and is not passed as a pointer. It was probably meant
to be a   *options |= EXIT_STATUS_EXACT   with the options argument
passed as a int*.

In any case, we do not want to run the code in main() that was supposed
to run if EXIT_STATUS_EXACT is set (but didn't since it is never added
to options); as far as I can tell, we only want to run that code when
the --exit-status/-e option is passed.

So I removed EXIT_STATUS_EXACT completely, and the useless assignment,
instead of fixing it since it was not used for anything else.

Useless assignment detected by clang-tidy.
2023-09-27 21:52:23 -05:00
623ace27e8 Remove a bunch of unused variables, and useless assignments
Detected by clang-tidy.
2023-09-27 21:50:44 -05:00
2b709727a9 Remove unused nref accumulator in block_bind_library
detected as a warning compiling jq with clang.
2023-09-27 21:50:44 -05:00
07149397d1 Reject U+001F in string literals (fix #2909) 2023-09-26 12:53:44 -05:00
e526df15fe Correct typo in README.md: compilation (#2912) 2023-09-26 19:49:07 +02:00
8f81668014 Fix the default colors to use 39, the default foreground color (#2904) 2023-09-22 09:18:41 +09:00
de1c10ca8a Bump docker/setup-qemu-action from 2 to 3 (#2900)
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 2 to 3.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-18 18:20:43 +09:00
a5afeea0a6 Bump docker/setup-buildx-action from 2 to 3 (#2901)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 2 to 3.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-18 18:20:20 +09:00
b0b7614daf Bump actions/checkout from 1 to 4 (#2902)
Bumps [actions/checkout](https://github.com/actions/checkout) from 1 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v1...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-18 18:19:51 +09:00
d3a356fa1c Fix checksum file spacing for shasum command (#2899) 2023-09-17 20:55:56 +09:00
24e6e6ea39 Bump actions/upload-artifact from 2 to 3
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-15 13:44:46 +02:00
fb493df018 Bump crazy-max/ghaction-import-gpg from 5 to 6
Bumps [crazy-max/ghaction-import-gpg](https://github.com/crazy-max/ghaction-import-gpg) from 5 to 6.
- [Release notes](https://github.com/crazy-max/ghaction-import-gpg/releases)
- [Commits](https://github.com/crazy-max/ghaction-import-gpg/compare/v5...v6)

---
updated-dependencies:
- dependency-name: crazy-max/ghaction-import-gpg
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-15 13:43:44 +02:00
f3a46a62de Bump docker/metadata-action from 4 to 5
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 4 to 5.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Upgrade guide](https://github.com/docker/metadata-action/blob/master/UPGRADE.md)
- [Commits](https://github.com/docker/metadata-action/compare/v4...v5)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-15 13:42:32 +02:00
e69398ea0f Bump docker/build-push-action from 4 to 5
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 4 to 5.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v4...v5)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-15 13:41:27 +02:00
dfd930f81a Bump docker/login-action from 2 to 3
Bumps [docker/login-action](https://github.com/docker/login-action) from 2 to 3.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-15 13:40:33 +02:00
4c6f6dc231 Add dependabot (#2889) 2023-09-15 20:09:07 +09:00
5ea1a4ab3b Disable core.autocrlf on Windows to prevent submodule diffs (fix #2886) (#2888) 2023-09-15 20:08:20 +09:00
7a72f58d2e Standarize arch types to AMD64 & ARM64 from index page download dropdown (#2884)
Standarize arch types to AMD64 & ARM64 from index page download
dropdown. These are missed from https://github.com/jqlang/jq/pull/2879.
2023-09-07 11:55:14 -07:00