1
0
mirror of https://github.com/stedolan/jq.git synced 2024-05-11 05:55:39 +00:00

1705 Commits

Author SHA1 Message Date
be437ec049 Add trim/0, ltrim/0 and rtrim/0 that trims leading and trailing whitespace (#3056) 2024-03-20 11:04:17 +01:00
81f4f883ac Remove non-standard pow10 filter in favor of exp10 (#3059)
The pow10 function was a glibc extension removed in 2.27. Use exp10 filter instead.
2024-03-20 18:23:08 +09:00
b6da71bf68 declare static arrays const where possible
Some static arrays that could be readonly memory were declared mutable.
2024-03-20 00:22:54 +01:00
bc96146825 builtin.c: jv2tm: fix UB and accept array inputs with not all the values
Now, time functions accept array inputs even if they don't have all the
elements, 0 will be assumed if a value is not present.

Also, jv2tm now properly clamps large number values to a signed 32-bit
integer and rejects nan.

Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65885
2024-03-19 02:01:53 +01:00
d69733154a Fix valgrind warning on aarch64/glibc 2.31
tests/mantest were failing on Raspberry Pi OS 11:

==10205== Conditional jump or move depends on uninitialised value(s)
==10205==    at 0x48806F8: checkfail (jq_test.c:58)
==10205==    by 0x48806F8: run_jq_tests (jq_test.c:99)
==10205==    by 0x4880FCB: jq_testsuite (jq_test.c:39)
==10205==    by 0x10B37F: main (main.c:576)
==10205==
==10205== Conditional jump or move depends on uninitialised value(s)
==10205==    at 0x4880714: checkfail (jq_test.c:58)
==10205==    by 0x4880714: run_jq_tests (jq_test.c:99)
==10205==    by 0x4880FCB: jq_testsuite (jq_test.c:39)
==10205==    by 0x10B37F: main (main.c:576)
2024-03-18 19:42:22 +01:00
10ea3c7d9f util.c: update strptime to revision 1.66 2024-03-18 19:10:21 +01:00
d777b6570c util.c: fix _WIN32 port of strptime (#3071)
In windows, time_t is a signed 32-bit integer type, so TIME_MAX needs to
be declared as INT32_MAX instead of INT64_MAX.

Also bump NetBSD's strptime to revision 1.65 from 1.63 to fix undefined
behaviour (signed integer overflow) bugs.

Related NetBSD problem report:
https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=58041

Noticed thanks to a compiler warning in the windows build CI.

I declared the d variable as  time_t  instead of as  unsigned  to ensure
that the signedness/size of  TIME_MAX-d  in the  sse > TIME_MAX-d check
is always correct, and to prevent -Wsign-compare warnings from
triggering in the windows build.
2024-03-18 17:22:59 +01:00
1411ce63d5 strftime/1: fix validation of non-string argument with number input
There was a incorrect else, that caused jq to not ensure that the
argument to strftime/1 is a string when the input is a number; this ends
up calling jv_string_value on a non-string value, which does not work,
and causes an assert failure.

Also fix same bug in strflocaltime/1.

Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67403
2024-03-15 14:30:23 +01:00
6f67bae60b EACH: fix leak when an error is triggered by non-last element of object
Object keys are strings, so they need to be freed.

Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=66070
2024-03-15 14:30:03 +01:00
c95b34ff82 remove repetitive words
Signed-off-by: wellweek <xiezitai@outlook.com>
2024-03-08 10:43:29 +01:00
ce0e788ce2 improve tonumber/0 performance by parsing input as number literal
Previously, the tonumber/0 filter parses the input as JSON values, but
this is less-performant on large non-number strings. Parsing the input
string as number literal fixes the performance issue. Also, this fix
changes the filter to reject numbers with white spaces.
2024-03-01 16:33:17 +01:00
913b26469f docs: Update chocolatey install command in default.yml (#3042) 2024-02-13 16:50:58 +01:00
fc9ecd71b3 tests: run tests in C locale
Fixes #3038
2024-02-10 18:23:50 +01:00
623d7a8bd6 src/jv_dtoa.c: Fix compiler error 2024-02-08 19:27:48 +01:00
6980cc9713 src/builtin.c: Add missing semicolon 2024-02-08 19:27:48 +01:00
92d35378ff Fix typo in documentation for map_values (#3035)
* Fix typo in documentation for map_values

"map_value(f)" -> "map_values(f)"

* map_value => map_values also in the devolpment manual

---------

Co-authored-by: Emanuele Torre <torreemanuele6@gmail.com>
2024-02-08 13:52:50 +00:00
54cc15c127 Add missing string color case
Colors have a color separate from object keys, so were not tested.
2024-02-08 00:11:54 -06:00
48df9bce0c Clear formatting immediately after colon 2024-02-08 00:11:54 -06:00
76853f6046 Fix tests and test indented printing 2024-02-08 00:11:54 -06:00
3c37522aaa Consistently reset color formatting in jv_dump_term
Before, arrays would not reset colors after `[` and `,`, but objects
would; it would reset colors twice before `]` and `}`; and some cases of
indentation would have colors applied. Now, colors are reset immediately
after any token that is colored, before any indentation. This makes the
formatting consistent, for the benefit of custom `JQ_COLORS`.
2024-02-08 00:11:54 -06:00
8a9a74dc12 License: Improve license statement.
The README mentions only the MIT license but in fact part of `jq`
is under ICU license and the documentation is under CC.

Also jq.spec claimed that it is licensed under BSD.
2024-02-06 14:51:46 +01:00
13e02ba348 docs: rename example jq utility to total
Apparently sum(1) is a BSD and GNU coreutil used to compute the
"16-bit BSD checksum" of a file.
2024-01-30 12:23:33 -06:00
bd3c828f98 libm.h+builtin.c: add and use LIBM_DA and LIBM_DA_NO macros
For functions that from one double return an array with two numbers.
2024-01-26 16:16:25 -06:00
605836b263 builtin.c: fix build with -Woverlength-strings
C99 only allows string literals long at most 4095 characters.
jq_builtins was a lot longer than that.

I rewrote all the optional libm error stubs in C so the value of
jq_builtins is not build dependent.

I replaced the command that generates builtin.inc with a POSIX compliant
od|sed command that encodes builtin.jq as a comma delimited list of
octal numbers (that can be embedded in C using a {} literal).

I also added -Woverlength-strings to AM_CFLAGS to verify that the
problem is fixed.

Fixes #1481
2024-01-26 16:16:25 -06:00
7bdc9a7a63 Makefile.am: fix lines indented with tabs instead of spaces 2024-01-26 16:13:35 -06:00
1daabc67ed typo contruct->construct (#3017) 2024-01-25 22:21:41 +01:00
c95f6ce700 Fix "in the a search" typo (#3015)
Fixes #3014
2024-01-24 23:34:41 +01:00
1f1e619f4e builtins: make ltrimstr and rtrimstr error for non-string inputs
Previously, ltrimstr/rtrimstr would just let the input pass through for
non-string inputs or arguments.

That was happening because, they were leaking the errors returned by
startswith/endswith treating them as if they were jv_false().
The leak was resolved by #2977 for 1.7.1

This patch rewrites ltrimstr and rtrimstr in jq, and makes them not
ignore startswith and endswith errors anymore.
2024-01-16 16:57:37 -06:00
71e7bcdfc1 Revert "lexer: temporarily revert #\ patch; keep CR in comment bug fix"
This reverts commit 5d95791a6795bfc44380c2e6e343ee66dd891e8b.
2024-01-16 16:08:03 -06:00
f954e82b14 Provide strptime implementation on all systems 2024-01-16 16:01:27 -06:00
f51e841fbc Bump jinja2 from 3.1.2 to 3.1.3 in /docs (#3009)
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.2 to 3.1.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/3.1.2...3.1.3)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-13 11:24:16 +09:00
2381d15d3e Bump the official-actions group with 2 updates (#2998)
* Bump the official-actions group with 2 updates

Bumps the official-actions group with 2 updates: [actions/upload-artifact](https://github.com/actions/upload-artifact) and [actions/download-artifact](https://github.com/actions/download-artifact).


Updates `actions/upload-artifact` from 3 to 4
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v3...v4)

Updates `actions/download-artifact` from 3 to 4
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: official-actions
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: official-actions
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix CI workflow for updating artifact actions to v4

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: itchyny <itchyny@cybozu.co.jp>
2024-01-09 09:33:26 +09:00
701c880805 Add jq 1.7.1 news item (#3004) 2023-12-26 15:22:39 +09:00
0b922725b1 Fix typo (#3002) 2023-12-24 11:22:20 +01:00
e011f003b6 Add jq 1.7.1 to webpage (#3003)
Make it the default download
2023-12-24 11:17:21 +01:00
56828b6d04 Always build Docker image on CI, group Dependabot PRs for official actions (#2997)
* Always build docker image on CI

* Revert "Bump actions/download-artifact from 3 to 4"

This reverts commit 9e2695e117aef8403fa0c5b32ee0f26cb442862a.

* Group Dependabot PRs for official actions
2023-12-19 17:54:53 +09:00
9e2695e117 Bump actions/download-artifact from 3 to 4
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 3 to 4.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-18 15:18:47 +01:00
95c50a2d0d Update signatures of 1.7.1
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-12-13 20:52:03 +01:00
71c2ab509a Merge pull request from GHSA-686w-5m7m-54vc
decNumberToString calls for a buffer that can hold a string of digits+14
characters, not a buffer of size digits+14.
We need to allocate an extra byte for the NUL byte.

-10E-1000010001, for example, will be stringified as -1.0E-1000010000
and decNumberToString will currently write an extra NUL byte after the
allocated buffer in the heap.

Originally reported by @SEU-SSL on GitHub.

Ref: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64574

Fixes GHSA-686w-5m7m-54vc
jq-1.7.1
2023-12-13 20:20:22 +01:00
c9a5156521 Merge pull request from GHSA-7hmr-442f-qc8j
The unit allocated for decNumberCompare was accidentally removed by
commit 680baeffeb7983e7570b5e68db07fe47f94db8c7 (PR #2804)

This caused a stack overflow when comparing a nan with a payload of 1000
or more.

This bug was found by OSS-fuzz.
Ref: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64771
Fixes GHSA-7hmr-442f-qc8j

It also fixes 1e999999999 > 1e-1147483646   triggering UBSAN errors
Fixes #2968
2023-12-13 20:17:17 +01:00
c5fd64ba97 jv_parse: let decNumberFromString/strtod parse complex nans as a NaN
Before this patch (when using decNumber), "Nan123" was parsed as a NaN,
only if the first n was uppercase.
2023-12-13 20:14:31 +01:00
dcf9a8e580 NEWS.md: fix markdown syntax; mention more bug fixs; list cves (#2987) 2023-12-13 18:39:24 +01:00
c3b14967d3 Update NEWS.md with changes since 1.7 (#2948) 2023-12-13 15:38:59 +00:00
5d95791a67 lexer: temporarily revert #\ patch; keep CR in comment bug fix
This commit temporarily reverts the commit that allows #\ "tcl-style"
comments everywhere and documents them, for the 1.7.1 patch release
cca1f7d18f2fa6721952645821ae429a0166d7e4.

\r is removed from the list of characters not allowed in a comment to
preserve that bugfix.
2023-12-13 16:38:18 +01:00
511d50b15b ltrimstr/1+rtrimstr/1: don't leak on invalid input or arguments
ltrimstr/rtrimstr was ignoring and leaking the error returned by
f_startswith()/f_endswith().

This also means that they just let the input pass through for non-string
inputs or arguments.

Only fix the leak for now; in the next release, #2969 will make them
rethrow the error returned by startswith/endswith.

Ref: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64946
2023-12-13 15:13:46 +01:00
7298972551 Makefile.am: don't use -lshlwapi to build jq on WIN32
It does not need it, only libjq needs this.
2023-12-13 13:08:57 +01:00
19453a60cf ci.yml: use sysctl -n hw.logicalcpu instead of nproc on macos
The "macos (arm64)" runner on github does not have nproc.

/Users/runner/work/_temp/f44f0d9f-19eb-4a23-860e-26533d7efdfa.sh: line 10: nproc: command not found
2023-12-11 22:02:42 +01:00
527c5abab1 jv_mem_calloc(): always call with (nmemb, size)
It does not matter much since they most likely just get multiplied
together, but some compilers would complain about this if these were
calls to calloc.
2023-12-11 20:47:32 +01:00
c2db51eeb3 shtest+configure: remove uses of non-portable/non-standard commands
Replace deprecated test(1) parentheses and -a logical operator with two
tests command.

Replace deprecated  tail -1  with  tail -n1.

Replace non-standard egrep(1) command with  grep -E  ; this also
prevents obsolescence warnings on GNU systems.
2023-12-11 20:46:54 +01:00
507eb8283c shtest: fix out-of-source tests
The locale test was using ./jq intead of $JQ.

I also removed the use of obsolete egrep instead of grep -E that
triggers warnings on GNU systems, and the use of deprecated head -1
instead of head -n1.

Also removed the unnecessary hiding of strptime/1 errors with
? // false.
2023-12-11 20:45:14 +01:00