Commit Graph
60 Commits
Author SHA1 Message Date
wxiaoguangandGitHub d242511e86 Remove legacy unmaintained packages, refactor to support change default locale (#19308)
Remove two unmaintained vendor packages `i18n` and `paginater`. Changes:
* Rewrite `i18n` package with a more clear fallback mechanism. Fix an unstable `Tr` behavior, add more tests.
* Refactor the legacy `Paginater` to `Paginator`, test cases are kept unchanged.

Trivial enhancement (no breaking for end users):
* Use the first locale in LANGS setting option as the default, add a log to prevent from surprising users.
2022-04-03 17:46:48 +08:00
909804b717 Bump goldmark to v1.4.11 (#19201)
* Bump goldmark to v1.4.11

* add testcase

Signed-off-by: Andrew Thornton <[email protected]>

Co-authored-by: Andrew Thornton <[email protected]>
2022-03-24 11:47:37 -04:00
b24e8d38af Support ignore all santize for external renderer (#18984)
* Support ignore all santize for external renderer

* Update docs

* Apply suggestions from code review

Co-authored-by: silverwind <[email protected]>

* Fix doc

Co-authored-by: silverwind <[email protected]>
Co-authored-by: 6543 <[email protected]>
2022-03-06 16:41:54 +08:00
6543andGitHub 54e9ee37a7 format with gofumpt (#18184)
* gofumpt -w -l .

* gofumpt -w -l -extra .

* Add linter

* manual fix

* change make fmt
2022-01-20 18:46:10 +01:00
zeripathandGitHub 5cb0c9aa0d Propagate context and ensure git commands run in request context (#17868)
This PR continues the work in #17125 by progressively ensuring that git
commands run within the request context.

This now means that the if there is a git repo already open in the context it will be used instead of reopening it.

Signed-off-by: Andrew Thornton <[email protected]>
2022-01-19 23:26:57 +00:00
GustedandGitHub ff2fd08228 Simplify parameter types (#18006)
Remove repeated type declarations in function definitions.
2021-12-20 04:41:31 +00:00
b748acf2a0 Fixed emoji alias not parsed in links (#16221)
* Do not skip links.

* Restrict text in links to emojis.

Co-authored-by: Lunny Xiao <[email protected]>
Co-authored-by: zeripath <[email protected]>
2021-12-15 23:49:12 +00:00
KN4CK3RandGitHub a09b40de8d Prevent double sanitize (#16386)
* Prevent double sanitize.
* Use SanitizeReaderToWriter.

At the moment `actualRender` uses `SanitizeReader` to sanitize the output. But `SanitizeReader` gets called in `markup.render` too so the output gets sanitized twice.

I moved the `SanitizeReader` call into `RenderRaw` because this method does not use `markup.render`. I would like to remove the `RenderRaw`/`RenderRawString` methods too because they are only called from tests, the fuzzer and the `/markup/raw` api endpoint. This endpoint is not in use so I think we could remove them. If we really in the future need a method to render markdown without PostProcessing we could achieve this with a more flexible `renderer.NeedPostProcess` method.
2021-11-19 18:46:47 +08:00
23bd7b1211 Add copy button to markdown code blocks (#17638)
* Add copy button to markdown code blocks

Done mostly in JS because I think it's better not to try getting buttons
past the markup sanitizer.

* add svg module tests

* fix sanitizer regexp

* remove outdated comment

* vertically center button in issue comments as well

* add comment to css

* fix undefined on view file line copy

* combine animation less files

* Update modules/markup/markdown/markdown.go

Co-authored-by: wxiaoguang <[email protected]>

* add test for different sizes

* add cloneNode and add tests for it

* use deep clone

* remove useless optional chaining

* remove the svg node cache

* unify clipboard copy string and i18n

* remove unused var

* remove unused localization

* minor css tweaks to the button

* comment tweak

* remove useless attribute

Co-authored-by: wxiaoguang <[email protected]>
2021-11-16 16:16:05 +08:00
wxiaoguangandGitHub d8e06a90f0 Open markdown image links in new window (#17287) 2021-10-11 20:12:06 +08:00
f2e7d5477f refactor: move from io/ioutil to io and os package (#17109)
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <[email protected]>

Co-authored-by: techknowlogick <[email protected]>
2021-09-22 13:38:34 +08:00
zeripathandGitHub 1fc7d6d6ad Fix table alignment in markdown (#16596)
Set the TableOptions in markdown to allow alignment of the tables to work correctly

Fix #15959

Signed-off-by: Andrew Thornton <[email protected]>
2021-08-02 17:11:18 +01:00
Lars LehtonenandGitHub 2344007fa3 modules/markup/markdown: fix dropped test error (#16438) 2021-07-18 14:09:34 +02:00
luzpazandGitHub e0296b6a6d Fix various documentation, user-facing, and source comment typos (#16367)
* Fix various doc, user-facing, and source comment typos

Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby`
2021-07-08 13:38:13 +02:00
c9c7afda1a Add sanitizer rules per renderer (#16110)
* Added sanitizer rules per renderer.

* Updated documentation.

Co-authored-by: techknowlogick <[email protected]>
2021-06-23 17:09:51 -04:00
KN4CK3RandGitHub 3607f79d78 Fixed assert statements. (#16089) 2021-06-07 07:27:09 +02:00
6021fbfe7a Make tasklist checkboxes clickable (#15791)
Co-authored-by: silverwind <[email protected]>
Co-authored-by: Lauris BH <[email protected]>
2021-05-23 17:14:03 +03:00
KN4CK3RandGitHub e818e9150f Fixed individual markdown tests. (#15802) 2021-05-09 15:29:49 +02:00
9d99f6ab19 Refactor renders (#15175)
* Refactor renders

* Some performance optimization

* Fix comment

* Transform reader

* Fix csv test

* Fix test

* Fix tests

* Improve optimaziation

* Fix test

* Fix test

* Detect file encoding with reader

* Improve optimaziation

* reduce memory usage

* improve code

* fix build

* Fix test

* Fix for go1.15

* Fix render

* Fix comment

* Fix lint

* Fix test

* Don't use NormalEOF when unnecessary

* revert change on util.go

* Apply suggestions from code review

Co-authored-by: zeripath <[email protected]>

* rename function

* Take NormalEOF back

Co-authored-by: zeripath <[email protected]>
2021-04-19 18:25:08 -04:00
Lunny XiaoandGitHub 66f0fd0959 Add NeedPostProcess for Parser interface to improve performance of csv render (#15153) 2021-04-13 15:06:31 +08:00
c680eb2cc7 Links in markdown should be absolute to the repository not the server (#15088)
* Links in markdown should be absolute to the repository not the server

Fix #15075

Signed-off-by: Andrew Thornton <[email protected]>

* match github

Signed-off-by: Andrew Thornton <[email protected]>

* add testcase

Signed-off-by: Andrew Thornton <[email protected]>

Co-authored-by: 6543 <[email protected]>
2021-04-11 00:26:28 +08:00
zeripathandGitHub 71aca93dec Remove extraneous logging (#15020)
Signed-off-by: Andrew Thornton <[email protected]>
2021-03-18 04:25:52 +01:00
ed31ddc29a Fix several render issues (#14986)
* Fix an issue with panics related to attributes
* Wrap goldmark render in a recovery function
* Reduce memory use in render emoji
* Use a pipe for rendering goldmark - still needs more work and a limiter

Signed-off-by: Andrew Thornton <[email protected]>
Co-authored-by: Lauris BH <[email protected]>
2021-03-16 00:20:05 +01:00
zeripathandGitHub 164e35ead3 Make sure sibling images get a link too (#14979)
* Make sure sibling images get a link too

Due a problem with the ast.Walker in the our transformer in goldmark
an image with a sibling image will not be transformed to gain a parent
link. This PR fixes this.

Fix #12925

Signed-off-by: Andrew Thornton <[email protected]>
2021-03-14 16:36:51 +00:00
41e8b834af Markdown task list improvements (#13952)
* Markdown task list improvements

- Remove `.ui` class and wrappers to prevent fomantic from messing with it.
- Change rendered HTML to match GitHub.
- Add custom styling for the checkboxes.

* fix unittest

Co-authored-by: Lauris BH <[email protected]>
Co-authored-by: techknowlogick <[email protected]>
2020-12-13 09:05:50 +08:00
b2bda11595 Fix markdown meta parsing (#12817)
* Fix meta parsing and add minimal test

Signed-off-by: jolheiser <[email protected]>

* Add license

Signed-off-by: jolheiser <[email protected]>

* Require first separator and add more tests

Signed-off-by: jolheiser <[email protected]>

* Make fmt

Signed-off-by: jolheiser <[email protected]>

* Go lets this work gracefully 🙏

Signed-off-by: jolheiser <[email protected]>

Co-authored-by: Lunny Xiao <[email protected]>
2020-09-12 21:48:47 -04:00
26c4a049da Issue templates directory (#11450)
* Issue templates

Signed-off-by: jolheiser <[email protected]>

* Add some comments, appease the linter

Signed-off-by: jolheiser <[email protected]>

* Add docs and re-use dir candidates

Signed-off-by: jolheiser <[email protected]>

* Add default labels to issue templates

Signed-off-by: jolheiser <[email protected]>

* Generate swagger

Signed-off-by: jolheiser <[email protected]>

* Suggested changes

Signed-off-by: jolheiser <[email protected]>

* Update issue.go

* Suggestions

Signed-off-by: jolheiser <[email protected]>

* Extract metadata from legacy if possible

Signed-off-by: jolheiser <[email protected]>

Co-authored-by: Lunny Xiao <[email protected]>
Co-authored-by: techknowlogick <[email protected]>
2020-09-11 10:48:39 -04:00
silverwindandGitHub e61c09ed73 Add loading spinners and mermaid error handling (#12358)
- Add loading spinners on editor and mermaid renderers
- Add error handling and inline error box for mermaid
- Fix Mermaid rendering by using the .init api
2020-08-04 20:56:37 +01:00
af7ffaa279 Server-side syntax highlighting for all code (#12047)
* Server-side syntax hilighting for all code

This PR does a few things:

* Remove all traces of highlight.js
* Use chroma library to provide fast syntax hilighting directly on the server
* Provide syntax hilighting for diffs
* Re-style both unified and split diffs views
* Add custom syntax hilighting styling for both regular and arc-green

Fixes #7729
Fixes #10157
Fixes #11825
Fixes #7728
Fixes #3872
Fixes #3682

And perhaps gets closer to #9553

* fix line marker

* fix repo search

* Fix single line select

* properly load settings

* npm uninstall highlight.js

* review suggestion

* code review

* forgot to call function

* fix test

* Apply suggestions from code review

suggestions from @silverwind thanks

Co-authored-by: silverwind <[email protected]>

* code review

* copy/paste error

* Use const for highlight size limit

* Update web_src/less/_repository.less

Co-authored-by: Lauris BH <[email protected]>

* update size limit to 1MB and other styling tweaks

* fix highlighting for certain diff sections

* fix test

* add worker back as suggested

Co-authored-by: silverwind <[email protected]>
Co-authored-by: Lauris BH <[email protected]>
2020-07-01 00:34:03 +03:00
2447ffc74a Disable all typographic replacements in markdown renderer (#11871)
* Disable all typographic replacements in markdown renderer

Previously we only disabled some of them. This disables all the default
replacements that goldmark's typographer extension offers, matching
GitHub's renderer.

Ref: https://github.com/yuin/goldmark#typographer-extension
Fixes: https://github.com/go-gitea/gitea/issues/11001

* remove typographer extension completely

* fix test

* really fix test

Co-authored-by: Lauris BH <[email protected]>
Co-authored-by: techknowlogick <[email protected]>
2020-06-13 02:10:17 -04:00
zeripathandGitHub 814ca9ffea Allow different HardBreaks settings for documents and comments (#11515)
GH has different HardBreaks behaviour for markdown comments and documents.

Comments have hard breaks and documents have soft breaks - therefore Gitea's rendering will always be different from GH's if we only provide one setting.

Here we split the setting in to two - one for documents and one for comments and other things.

Signed-off-by: Andrew Thornton [email protected]

Changes to index.js as per @silverwind 
Co-authored-by: silverwind <[email protected]>

Changes to docs as per @guillep2k 
Co-authored-by: guillep2k <[email protected]>
2020-05-24 09:14:26 +01:00
zeripathandGitHub 742e26f5a5 Prevent 500 with badly formed task list (#11328)
Fix #11317

Signed-off-by: Andrew Thornton <[email protected]>
2020-05-11 00:14:49 +01:00
zeripathandGitHub ce66ca7f9f Restore checkbox rendering and prevent poor sanitization of spans (#11277)
* Add test

Signed-off-by: Andrew Thornton <[email protected]>

* Restore checkbox rendering and prevent poor sanitization of spans

Signed-off-by: Andrew Thornton <[email protected]>

* Also fix preview context

Signed-off-by: Andrew Thornton <[email protected]>

* Also fix preview context

Signed-off-by: Andrew Thornton <[email protected]>
2020-05-03 17:17:24 -03:00
zeripathandGitHub 0e799c26ba Prevent panic during wrappedConn close at hammertime (#11219)
* Prevent panic during wrappedConn close at hammertime

Signed-off-by: Andrew Thornton <[email protected]>

* Update modules/graceful/server.go

* Fix extraneous debug in goldmark.go

Signed-off-by: Andrew Thornton <[email protected]>
2020-04-26 21:01:06 -03:00
9f959ac064 Make TaskCheckBox render correctly (#11214)
* Fix checkbox rendering

Signed-off-by: Andrew Thornton <[email protected]>

* Normalize checkbox rendering

Signed-off-by: Andrew Thornton <[email protected]>

* set the checkboxes to readonly instead of disabled

Signed-off-by: Andrew Thornton <[email protected]>

Co-authored-by: Lauris BH <[email protected]>
2020-04-26 02:09:08 -03:00
zeripathandGitHub 812cfd0ad9 Use markdown frontmatter to provide Table of contents, language and frontmatter rendering (#11047)
* Add control for the rendering of the frontmatter
* Add control to include a TOC
* Add control to set language - allows control of ToC header and CJK glyph choice.

Signed-off-by: Andrew Thornton [email protected]
2020-04-24 14:22:36 +01:00
silverwindandGitHub 7e20f1cb5b enable ENABLE_HARD_LINE_BREAK by default (#11162)
enable ENABLE_HARD_LINE_BREAK by default

This matches GitHub's rendering.

Fix #11155
2020-04-21 23:13:56 +01:00
zeripathandGitHub 0be25e2150 Handle yaml frontmatter (#11016)
Add goldmark-meta to render yaml frontmatter as a table

Fix #5377

Signed-off-by: Andrew Thornton <[email protected]>
2020-04-09 11:54:50 +01:00
zeripathandGitHub cf4f2cfa3e Fix internal server error on checkboxes (#10845)
Annoyingly goldmarks SetAttributeString requires that
the value of the attribute is still a []byte but does
not make it clear in the documentation.

Signed-off-by: Andrew Thornton <[email protected]>
2020-03-26 23:47:34 +00:00
zeripathandGitHub f036de5a94 Completely fix task-list checkbox styling (#10798)
Continuing on from #10668 this PR makes goldmark apply the task-list styling to task-lists.
2020-03-22 22:25:38 +00:00
b5f28d154d Don't manually replace whitespace during render (#10291)
* Don't manually replace whitespace during render

For historical reasons Gitea manually alters the urlPrefix and replaces
a whitespace with a +. This Works for URLs, but we're also passing
urlPrefix to git calls and adding the + is breaking the tree path.

Goldmark will automatically convert a white space to the proper %20, so
we should leave the string as is which lets us pass it to git unmodified
and then let Goldmark fix it.

Also fixed separate bug in URLJoin I noticed while testing where it will
silently discard sections of a path that have # in them (possibly
others). We should just escape it first.

Fixes 10156

* Escape elems as well

* Revert "Escape elems as well"

This reverts commit 8bf49596fe.

* restart ci

* remove changes to URLJoin

* restart ci

Co-authored-by: techknowlogick <[email protected]>
2020-02-17 07:46:51 +02:00
53f9dbfc7b Don't convert ellipsis in markdown (#9905)
* Don't convert ellipsis

Signed-off-by: jolheiser <[email protected]>

* Formatting

Co-Authored-By: zeripath <[email protected]>

Co-authored-by: Lunny Xiao <[email protected]>
Co-authored-by: Antoine GIRARD <[email protected]>
Co-authored-by: zeripath <[email protected]>
2020-01-22 16:30:20 +01:00
7d7ab1eeae Issue/PR Context Popups (#9822)
* Add data-index attribute to issue anchors

Signed-off-by: jolheiser <[email protected]>

* Init JS

Signed-off-by: jolheiser <[email protected]>

* Add required data to anchor

Signed-off-by: jolheiser <[email protected]>

* Finish popup

Signed-off-by: jolheiser <[email protected]>

* Revert changes to html.go

Signed-off-by: jolheiser <[email protected]>

* Better octicon contexts

Signed-off-by: jolheiser <[email protected]>

* Split out popup function for re-use

Signed-off-by: jolheiser <[email protected]>

* Style changes, test fixes, and cross-reference support

Signed-off-by: jolheiser <[email protected]>

* Prefer em to px

Signed-off-by: jolheiser <[email protected]>

* Move label margin to base CSS

Signed-off-by: jolheiser <[email protected]>

* Move JS to separate file.

Signed-off-by: jolheiser <[email protected]>

* Move JS to features and fix module

Signed-off-by: jolheiser <[email protected]>

* Remove query-string and hash

Co-Authored-By: silverwind <[email protected]>

Co-authored-by: Lauris BH <[email protected]>
Co-authored-by: Antoine GIRARD <[email protected]>
Co-authored-by: silverwind <[email protected]>
Co-authored-by: Lunny Xiao <[email protected]>
Co-authored-by: zeripath <[email protected]>
2020-01-19 23:39:21 -05:00
f740943df4 Fix markdown anchor links (#9673)
Co-authored-by: Lauris BH <[email protected]>
2020-01-16 12:23:48 +01:00
zeripathandLauris BH 27757714d0 Change markdown rendering from blackfriday to goldmark (#9533)
* Move to goldmark

Markdown rendering moved from blackfriday to the goldmark.

Multiple subtle changes required to the goldmark extensions to keep
current rendering and defaults.

Can go further with goldmark linkify and have this work within markdown
rendering making the link processor unnecessary.

Need to think about how to go about allowing extensions - at present it
seems that these would be hard to do without recompilation.

* linter fixes

Co-authored-by: Lauris BH <[email protected]>
2019-12-31 03:53:28 +02:00
John Olheiserandzeripath e6ceb6880a Prefix all user-generated IDs in markup (#9477)
* Prefix all user-generated IDs in markup
* Add user-content- to IDs in unit-tests
* fixup markdown_test.go
* update the hrefs for the wiki test
* Add blackfriday extension regex

Signed-off-by: jolheiser <[email protected]>
2019-12-23 22:38:50 +00:00
guillep2kandAntoine GIRARD 7b97e04555 Convert EOL to UNIX-style to render MD properly (#8925)
* Convert EOL to UNIX-style to render MD properly

* Update modules/markup/markdown/markdown.go

Co-Authored-By: zeripath <[email protected]>

* Fix lint optimization

* Check for empty content before conversion

* Update modules/util/util.go

Co-Authored-By: zeripath <[email protected]>

* Improved checks and tests

* Add paragraph render test

* Improve speed even more, improve tests

* Small improvement by @gary-kim

* Fix test for DOS

* More improvements

* Restart CI
2019-11-13 03:27:11 +01:00
Lauris BHandzeripath 086a46994a Rewrite markdown rendering to blackfriday v2 and rewrite orgmode rendering to go-org (#8560)
* Rewrite markdown rendering to blackfriday v2.0

* Fix style

* Fix go mod with golang 1.13

* Fix blackfriday v2 import

* Inital orgmode renderer migration to go-org

* Vendor go-org dependency

* Ignore errors :/

* Update go-org to latest version

* Update test

* Fix go-org test

* Remove unneeded code

* Fix comments

* Fix markdown test

* Fix blackfriday regression rendering HTML block
2019-10-31 01:06:25 +00:00
Gary KimandLunny Xiao 7eed11e5e9 Check commit message hashes before making links (#7713)
* Check commit message hashes before making links

Previously, when formatting commit messages, anything
that looked like SHA1 hashes was turned into a link
using regex. This meant that certain phrases or numbers
such as `777777` or `deadbeef` could be recognized as a commit
even if the repository has no commit with those hashes.

This change will make it so that anything that looks
like a SHA1 hash using regex will then also be checked
to ensure that there is a commit in the repository
with that hash before making a link.

Signed-off-by: Gary Kim <[email protected]>

* Use gogit to check if commit exists

This commit modifies the commit hash check
in the render for commit messages to use
gogit for better performance.

Signed-off-by: Gary Kim <[email protected]>

* Make code cleaner

Signed-off-by: Gary Kim <[email protected]>

* Use rev-parse to check if commit exists

Signed-off-by: Gary Kim <[email protected]>

* Add and modify tests for checking hashes in html link rendering

Signed-off-by: Gary Kim <[email protected]>

* Return error in sha1CurrentPatternProcessor

Co-Authored-By: mrsdizzie <[email protected]>

* Import Gitea log module

Signed-off-by: Gary Kim <[email protected]>

* Revert "Return error in sha1CurrentPatternProcessor"

This reverts commit 28f561cac4.

Signed-off-by: Gary Kim <[email protected]>

* Add debug logging to sha1CurrentPatternProcessor

This will log errors by the git command run in
sha1CurrentPatternProcessor if the error is one
that was unexpected.

Signed-off-by: Gary Kim <[email protected]>
2019-08-14 16:04:55 +08:00
Christian Muehlhaeuserandzeripath 54d96c79b5 Removed unnecessary conversions (#7557)
No need to convert to the same type.
2019-07-23 19:50:39 +01:00