Change all license headers to comply with REUSE specification.
Fix#16132
Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <[email protected]>
Continues on from #19202.
Following the addition of pprof labels we can now more easily understand the relationship between a goroutine and the requests that spawn them.
This PR takes advantage of the labels and adds a few others, then provides a mechanism for the monitoring page to query the pprof goroutine profile.
The binary profile that results from this profile is immediately piped in to the google library for parsing this and then stack traces are formed for the goroutines.
If the goroutine is within a context or has been created from a goroutine within a process context it will acquire the process description labels for that process.
The goroutines are mapped with there associate pids and any that do not have an associated pid are placed in a group at the bottom as unbound.
In this way we should be able to more easily examine goroutines that have been stuck.
A manager command `gitea manager processes` is also provided that can export the processes (with or without stacktraces) to the command line.
Signed-off-by: Andrew Thornton <[email protected]>
* Attempt to prevent the deadlock in the QueueDiskChannel Test again
This time we're going to adjust the pause tests to only test the right
flag.
* Only switch off pushback once we know that we are not pushing anything else
* Ensure full redirection occurs
* More nicely handle a closed datachan
* And handle similar problems in queue_channel_test
Signed-off-by: Andrew Thornton <[email protected]>
Implements the Pausable interface on WrappedQueues and PersistableChannelUniqueQueues
Reference #15928
Signed-off-by: Andrew Thornton [email protected]
* Start adding mechanism to return unhandled data
Signed-off-by: Andrew Thornton <[email protected]>
* Create pushback interface
Signed-off-by: Andrew Thornton <[email protected]>
* Add Pausable interface to WorkerPool and Manager
Signed-off-by: Andrew Thornton <[email protected]>
* Implement Pausable and PushBack for the bytefifos
Signed-off-by: Andrew Thornton <[email protected]>
* Implement Pausable and Pushback for ChannelQueues and ChannelUniqueQueues
Signed-off-by: Andrew Thornton <[email protected]>
* Wire in UI for pausing
Signed-off-by: Andrew Thornton <[email protected]>
* add testcases and fix a few issues
Signed-off-by: Andrew Thornton <[email protected]>
* fix build
Signed-off-by: Andrew Thornton <[email protected]>
* prevent "race" in the test
Signed-off-by: Andrew Thornton <[email protected]>
* fix jsoniter mismerge
Signed-off-by: Andrew Thornton <[email protected]>
* fix conflicts
Signed-off-by: Andrew Thornton <[email protected]>
* fix format
Signed-off-by: Andrew Thornton <[email protected]>
* Add warnings for no worker configurations and prevent data-loss with redis/levelqueue
Signed-off-by: Andrew Thornton <[email protected]>
* Use StopTimer
Signed-off-by: Andrew Thornton <[email protected]>
Co-authored-by: Lauris BH <[email protected]>
Co-authored-by: 6543 <[email protected]>
Co-authored-by: techknowlogick <[email protected]>
Co-authored-by: wxiaoguang <[email protected]>
* move shutdownfns, terminatefns and hammerfns out of separate goroutines
Coalesce the shutdownfns etc into a list of functions that get run at shutdown
rather then have them run at goroutines blocked on selects.
This may help reduce the background select/poll load in certain
configurations.
* The LevelDB queues can actually wait on empty instead of polling
Slight refactor to cause leveldb queues to wait on empty instead of polling.
* Shutdown the shadow level queue once it is empty
* Remove bytefifo additional goroutine for readToChan as it can just be run in run
* Remove additional removeWorkers goroutine for workers
* Simplify the AtShutdown and AtTerminate functions and add Channel Flusher
* Add shutdown flusher to CUQ
* move persistable channel shutdown stuff to Shutdown Fn
* Ensure that UPCQ has the correct config
* handle shutdown during the flushing
* reduce risk of race between zeroBoost and addWorkers
* prevent double shutdown
Signed-off-by: Andrew Thornton <[email protected]>
* Upgrade levelqueue to version 0.2.0
This adds functionality for Unique Queues
* Add UniqueQueue interface and functions to create them
* Add UniqueQueue implementations
* Move TestPullRequests over to use UniqueQueue
* Reduce code duplication
* Add bytefifos
* Ensure invalid types are logged
* Fix close race in PersistableChannelQueue Shutdown
* Make WorkerPools and Queues flushable
Adds Flush methods to Queues and the WorkerPool
Further abstracts the WorkerPool
Adds a final step to Flush the queues in the defer from PrintCurrentTest
Fixes an issue with Settings inheritance in queues
Signed-off-by: Andrew Thornton <[email protected]>
* Change to for loop
* Add IsEmpty and begin just making the queues composed WorkerPools
* subsume workerpool into the queues and create a flushable interface
* Add manager command
* Move flushall to queue.Manager and add to testlogger
* As per @guillep2k
* as per @guillep2k
* Just make queues all implement flushable and clean up the wrapped queue flushes
* cope with no timeout
Co-authored-by: Lauris BH <[email protected]>
flynnnnnnnnnn
zeripath