Commit Graph
138 Commits
Author SHA1 Message Date
d2d99a25b7 Fix NPE in fuzzer (#16680) (#16682)
The fuzzer found an issue with the issue pattern processor where there is a spurious
path.Clean which does not need to be there. This PR also sets the default AppURL for
the fuzzer too.

Signed-off-by: Andrew Thornton <[email protected]>

Co-authored-by: zeripath <[email protected]>
2021-08-13 08:51:13 +03:00
zeripathandGitHub f9120092c1 Fix table alignment in markdown (#16596) (#16602)
Backport #16596

Set the TableOptions in markdown to allow alignment of the tables to work correctly

Fix #15959

Signed-off-by: Andrew Thornton <[email protected]>
2021-08-04 00:16:00 +01:00
46a4c6835d Fix external renderer (#16401)
* fix external renderer

* use GBackground context as fallback

* no fallback, return error

Co-authored-by: Lauris BH <[email protected]>
2021-07-12 17:13:59 -04:00
6543andGitHub 91162bbaea Update bluemonday to v1.0.15 (#16379)
* update github.com/microcosm-cc/bluemonday

* add exec flag to contrib/update_dependencies.sh

* Fix TESTS
2021-07-09 03:30:31 +02:00
luzpazandGitHub e0296b6a6d Fix various documentation, user-facing, and source comment typos (#16367)
* Fix various doc, user-facing, and source comment typos

Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby`
2021-07-08 13:38:13 +02:00
32fd11395b Fix relative links in postprocessed images (#16334)
If a pre-post-processed file contains relative img tags these need to be updated
and joined correctly with the prefix. Finally, the node attributes need to be updated.

Fix #16308

Signed-off-by: Andrew Thornton <[email protected]>
Co-authored-by: 6543 <[email protected]>
2021-07-04 10:26:04 +01:00
zeripathandGitHub 302e8b6d02 Prevent zombie processes (#16314)
Unfortunately go doesn't always ensure that execd processes are completely
waited for. On linux this means that zombie processes can occur.

This PR ensures that these are waited for by using signal notifier in serv and
passing a context elsewhere.

Signed-off-by: Andrew Thornton <[email protected]>
2021-06-30 22:07:23 +02:00
6543andGitHub 65548359cc Add custom emoji support (#16004) 2021-06-29 16:28:38 +02:00
c9c7afda1a Add sanitizer rules per renderer (#16110)
* Added sanitizer rules per renderer.

* Updated documentation.

Co-authored-by: techknowlogick <[email protected]>
2021-06-23 17:09:51 -04:00
f2babf3346 Add code block highlight to orgmode back (#14222)
Fix missed orgmode code block hightlight

Co-authored-by: zeripath <[email protected]>
2021-06-23 22:45:47 +02:00
zeripathandGitHub d55b5eb0d3 Use html.Parse rather than html.ParseFragment (#16223)
* Use html.Parse rather than html.ParseFragment
  There have been a few issues with html.ParseFragment - just use html.Parse instead.

* Skip document node

Signed-off-by: Andrew Thornton <[email protected]>
2021-06-22 00:12:22 +02:00
196593e2e9 More efficiently parse shas for shaPostProcessor (#16101)
* More efficiently parse shas for shaPostProcessor

The shaPostProcessor currently repeatedly calls git rev-parse --verify on both backends
which is fine if there is only one thing that matches a sha - however if there are
multiple things then this becomes wildly inefficient.

This PR provides functions for both backends which are much faster to use.

Fix #16092

* Add ShaExistCache to RenderContext

Signed-off-by: Andrew Thornton <[email protected]>
Co-authored-by: 6543 <[email protected]>
2021-06-21 00:39:12 +02:00
zeripathandGitHub 0db1048c3a Run processors on whole of text (#16155)
There is an inefficiency in the design of our processors which means that Emoji
and other processors run in order n^2 time.

This PR forces the processors to process the entirety of text node before passing
back up. The fundamental inefficiency remains but it should be significantly
ameliorated.

Signed-off-by: Andrew Thornton <[email protected]>
2021-06-17 11:35:05 +01:00
zeripathandGitHub f4d3bf7867 issue-keyword class is being incorrectly stripped off spans (#16163)
Bluemonday sanitizer regexp rules are not additive, so the addition of the icons,
emojis and chroma syntax policy has led to this being stripped.

Signed-off-by: Andrew Thornton <[email protected]>
2021-06-16 03:02:03 +02:00
KN4CK3RandGitHub 21cde5c439 Fix data URI scramble (#16098)
* Removed unused method.

* No prefix for data uris.

* Added test to prevent regressions.
2021-06-07 18:55:26 +02:00
KN4CK3RandGitHub 3607f79d78 Fixed assert statements. (#16089) 2021-06-07 07:27:09 +02:00
Lunny XiaoandGitHub b6762e2306 Fix regression of renderer (#16091)
* Fix regression of renderer

* Fix render setting load twice bug
2021-06-06 18:50:07 -04:00
6021fbfe7a Make tasklist checkboxes clickable (#15791)
Co-authored-by: silverwind <[email protected]>
Co-authored-by: Lauris BH <[email protected]>
2021-05-23 17:14:03 +03:00
KN4CK3RandGitHub e818e9150f Fixed individual markdown tests. (#15802) 2021-05-09 15:29:49 +02:00
silverwindandGitHub d4f28fd4ad Fix URL of gitea emoji (#15770)
Fixes regression from #15219
2021-05-07 17:34:33 +02:00
9d99f6ab19 Refactor renders (#15175)
* Refactor renders

* Some performance optimization

* Fix comment

* Transform reader

* Fix csv test

* Fix test

* Fix tests

* Improve optimaziation

* Fix test

* Fix test

* Detect file encoding with reader

* Improve optimaziation

* reduce memory usage

* improve code

* fix build

* Fix test

* Fix for go1.15

* Fix render

* Fix comment

* Fix lint

* Fix test

* Don't use NormalEOF when unnecessary

* revert change on util.go

* Apply suggestions from code review

Co-authored-by: zeripath <[email protected]>

* rename function

* Take NormalEOF back

Co-authored-by: zeripath <[email protected]>
2021-04-19 18:25:08 -04:00
Lunny XiaoandGitHub 66f0fd0959 Add NeedPostProcess for Parser interface to improve performance of csv render (#15153) 2021-04-13 15:06:31 +08:00
c680eb2cc7 Links in markdown should be absolute to the repository not the server (#15088)
* Links in markdown should be absolute to the repository not the server

Fix #15075

Signed-off-by: Andrew Thornton <[email protected]>

* match github

Signed-off-by: Andrew Thornton <[email protected]>

* add testcase

Signed-off-by: Andrew Thornton <[email protected]>

Co-authored-by: 6543 <[email protected]>
2021-04-11 00:26:28 +08:00
b9ed3cbc26 Upgrade to bluemonday 1.0.7 (#15379)
* Upgrade to bluemonday 1.0.7

Fix #15349

Signed-off-by: Andrew Thornton <[email protected]>

* resolve unit test

Co-authored-by: techknowlogick <[email protected]>
2021-04-09 19:13:06 -04:00
6543andGitHub 9c4601bdf8 Code Formats, Nits & Unused Func/Var deletions (#15286)
* _ to unused func options

* rm useless brakets

* rm trifial non used models functions

* rm dead code

* rm dead global vars

* fix routers/api/v1/repo/issue.go

* dont overload import module
2021-04-09 09:40:34 +02:00
zeripathandGitHub 04196b7658 Update to bluemonday-1.0.6 (#15294)
Signed-off-by: Andrew Thornton <[email protected]>
2021-04-05 17:38:31 -04:00
KN4CK3RandGitHub 0c6137617f Add Tabular Diff for CSV files (#14661)
Implements request #14320 The rendering of CSV files does match the diff style.

* Moved CSV logic into base package.

* Added method to create a tabular diff.

* Added CSV compare context.

* Added CSV diff template.

* Use new table style in CSV markup.

* Added file size limit for CSV rendering.

* Display CSV parser errors in diff.

* Lazy read single file.

* Lazy read rows for full diff.

* Added unit tests for various CSV changes.
2021-03-29 22:44:28 +02:00
zeripathandGitHub b68eb54f95 Clusterfuzz found another way (#15160)
Clusterfuzz found another way so I found another way to stop it

Signed-off-by: Andrew Thornton <[email protected]>
2021-03-26 01:21:00 +02:00
zeripathandGitHub a587a28434 Fix another clusterfuzz identified issue (#15096)
* Fix another clusterfuzz identified issue

Signed-off-by: Andrew Thornton <[email protected]>
2021-03-22 02:45:01 +01:00
zeripathandGitHub c0c052bdbb another clusterfuzz spotted issue (#15032)
Signed-off-by: Andrew Thornton <[email protected]>
2021-03-18 22:37:11 +01:00
zeripathandGitHub 71aca93dec Remove extraneous logging (#15020)
Signed-off-by: Andrew Thornton <[email protected]>
2021-03-18 04:25:52 +01:00
ed31ddc29a Fix several render issues (#14986)
* Fix an issue with panics related to attributes
* Wrap goldmark render in a recovery function
* Reduce memory use in render emoji
* Use a pipe for rendering goldmark - still needs more work and a limiter

Signed-off-by: Andrew Thornton <[email protected]>
Co-authored-by: Lauris BH <[email protected]>
2021-03-16 00:20:05 +01:00
zeripathandGitHub 164e35ead3 Make sure sibling images get a link too (#14979)
* Make sure sibling images get a link too

Due a problem with the ast.Walker in the our transformer in goldmark
an image with a sibling image will not be transformed to gain a parent
link. This PR fixes this.

Fix #12925

Signed-off-by: Andrew Thornton <[email protected]>
2021-03-14 16:36:51 +00:00
zeripathandGitHub aa4f9180e4 Clarify the suffices and prefixes of setting.AppSubURL and setting.AppURL (#12999)
Also removes some unnecessary uses of fmt.Sprintf and adds documentation
strings

Signed-off-by: Andrew Thornton <[email protected]>
2021-02-19 22:36:43 +01:00
zeripathandGitHub 7ab6c77b41 Remove NULs byte arrays passed to PostProcess (#14587)
PostProcess is supposed to be parsing and handling HTML
fragments, but on fuzzing it appears that there is a weird
issue with NUL elements that could cause a memory address
error in downstream libraries.

The simplest solution is to strip out the weird NULs - they
should not be there in any case and would be stripped out
anyway.

Signed-off-by: Andrew Thornton <[email protected]>
2021-02-18 02:32:14 +01:00
zeripathandGitHub 172229966c Prevent panic on fuzzer provided string (#14405)
* Prevent panic on fuzzer provided string

The fuzzer has found that providing a <body> tag with an attribute to
PostProcess causes a panic. This PR removes any rendered html or body
tags from the output.

Signed-off-by: Andrew Thornton <[email protected]>

* Placate lint

* placate lint again

Signed-off-by: Andrew Thornton <[email protected]>

* minor cleanup

Signed-off-by: Andrew Thornton <[email protected]>
2021-01-20 23:10:50 +08:00
Nuno SilvaandGitHub 44ff1d6a1e Render links for commit hashes followed by comma (#14224)
Regex test cases: https://regex101.com/r/mVbPxM/2/

fixes #14223
2021-01-03 23:11:10 +08:00
11555d850b Fix bug of link query order on markdown render (#14156)
* Fix bug of link query order on markdown render

* Fix bluemonday bug and fix one wrong test

Co-authored-by: 6543 <[email protected]>
2020-12-29 00:28:27 +08:00
34df4e5df5 Add mentionable teams to tributeValues and change team mention rules to gh's style (#13198)
* Add mentionable teams to tributeValues

Signed-off-by: a1012112796 <[email protected]>

* Apply suggestions from code review

Co-authored-by: silverwind <[email protected]>

* Change team mention rules to gh's style

* use org's avator as team avator in ui

Signed-off-by: a1012112796 <[email protected]>

* Update modules/markup/html.go

* Update models/issue.go

Co-authored-by: Lauris BH <[email protected]>

* Update models/issue.go

* fix a small nit and update test code

Co-authored-by: silverwind <[email protected]>
Co-authored-by: Lauris BH <[email protected]>
Co-authored-by: 6543 <[email protected]>
2020-12-21 16:39:28 +01:00
41e8b834af Markdown task list improvements (#13952)
* Markdown task list improvements

- Remove `.ui` class and wrappers to prevent fomantic from messing with it.
- Change rendered HTML to match GitHub.
- Add custom styling for the checkboxes.

* fix unittest

Co-authored-by: Lauris BH <[email protected]>
Co-authored-by: techknowlogick <[email protected]>
2020-12-13 09:05:50 +08:00
mrsdizzieandGitHub 3512c7e40f Fix crash in short link processor (#13839)
Fixes #13819
2020-12-04 03:01:42 +01:00
Jimmy PraetandGitHub 056b8f5b52 Clickable links in pull request (and issue) titles (#13695)
* clickable links in pull request (and issue) titles #13658

reuses the existing logic to render clickable links in commit messages

* dedicated RenderIssueTitle function

applied patch from @mrsdizzie
2020-12-03 11:50:47 +01:00
kolaenteandGitHub 64133126cd Update golangci-lint to version 1.31.0 (#13102)
This PR updates golangci-lint to the latest version 1.31.0.

The upgrade introduced a new check for which I've fixed or disabled most cases.

Signed-off-by: kolaente <[email protected]>
2020-10-11 21:27:20 +01:00
1827f892de fix: media links in org files not liked to media files (#12997)
* fix: media links in org files not liked to media files

* fix: write directly to io.Writer r

as suggested by code review

Co-authored-by: zeripath <[email protected]>

Co-authored-by: zeripath <[email protected]>
2020-10-01 11:22:34 -04:00
b2bda11595 Fix markdown meta parsing (#12817)
* Fix meta parsing and add minimal test

Signed-off-by: jolheiser <[email protected]>

* Add license

Signed-off-by: jolheiser <[email protected]>

* Require first separator and add more tests

Signed-off-by: jolheiser <[email protected]>

* Make fmt

Signed-off-by: jolheiser <[email protected]>

* Go lets this work gracefully 🙏

Signed-off-by: jolheiser <[email protected]>

Co-authored-by: Lunny Xiao <[email protected]>
2020-09-12 21:48:47 -04:00
zeripathandGitHub f91bb745d6 Don't replace underscores in auto-generated IDs in goldmark (#12805)
Fix #12196

Signed-off-by: Andrew Thornton <[email protected]>
2020-09-12 12:00:48 -04:00
26c4a049da Issue templates directory (#11450)
* Issue templates

Signed-off-by: jolheiser <[email protected]>

* Add some comments, appease the linter

Signed-off-by: jolheiser <[email protected]>

* Add docs and re-use dir candidates

Signed-off-by: jolheiser <[email protected]>

* Add default labels to issue templates

Signed-off-by: jolheiser <[email protected]>

* Generate swagger

Signed-off-by: jolheiser <[email protected]>

* Suggested changes

Signed-off-by: jolheiser <[email protected]>

* Update issue.go

* Suggestions

Signed-off-by: jolheiser <[email protected]>

* Extract metadata from legacy if possible

Signed-off-by: jolheiser <[email protected]>

Co-authored-by: Lunny Xiao <[email protected]>
Co-authored-by: techknowlogick <[email protected]>
2020-09-11 10:48:39 -04:00
9fdb4f887b Update to go-org 1.3.2 (#12728)
* Update to go-org 1.3.2

Fix #12727

Signed-off-by: Andrew Thornton <[email protected]>

* Fix unit test

Signed-off-by: Andrew Thornton <[email protected]>

Co-authored-by: Lunny Xiao <[email protected]>
2020-09-05 11:45:10 -04:00
ee047312a1 Fix emoji replacements, make emoji images consistent (#12567)
- Fix emoji not being replaced in issue title change text
- Make the image attributes consistent, add alt, remove align

Co-authored-by: zeripath <[email protected]>
Co-authored-by: techknowlogick <[email protected]>
2020-08-23 21:44:53 -04:00
74bd9691c6 Re-attempt to delete temporary upload if the file is locked by another process (#12447)
Replace all calls to os.Remove/os.RemoveAll by retrying util.Remove/util.RemoveAll and remove circular dependencies from util.

Fix #12339

Signed-off-by: Andrew Thornton <[email protected]>
Co-authored-by: silverwind <[email protected]>
2020-08-11 21:05:34 +01:00