(cherry picked from commit 57e597bf7e)
(cherry picked from commit 643a2b0e81)
(cherry picked from commit f10faffb4f)
(cherry picked from commit b440c5767e)
[TESTS] verify facts for the admin storage documentation (squash)
(cherry picked from commit d83d8ce57b)
(cherry picked from commit d8855ef27c)
(cherry picked from commit 11230466ec)
(cherry picked from commit b2cdd9d971)
(cherry picked from commit a0a5e78524)
(cherry picked from commit 846413110f)
(cherry picked from commit 72b92d5a78)
(cherry picked from commit 7e039a9427)
(cherry picked from commit 227d42a1b6)
(cherry picked from commit 6488950a9b)
(cherry picked from commit 0285c99774)
(cherry picked from commit 0527bb20db)
(cherry picked from commit b4d3fd43e3)
(cherry picked from commit b45db64300)
(cherry picked from commit 722ab37611)
(cherry picked from commit ff45632b42)
(cherry picked from commit 5144ae2aa7)
(cherry picked from commit f51438cd68)
(cherry picked from commit 5ec304e5a1)
(cherry picked from commit 440b8c1af1)
(cherry picked from commit e5bbc626c1)
(cherry picked from commit a996fe0fe4)
Follow-up to d58c542579 for Forgejo.
By default, Gitea does not select any map service that can be used
to introduce a 'Show this place on a map' button in the location
field of a user profile. Before I tried upstreaming this change to
Gitea, this was the case in Forgejo. This patch essentially recovers
this functionality, which is nice for public-facing instances and
communities.
Links to original PRs:
- https://codeberg.org/forgejo/forgejo/pulls/1076
- https://github.com/go-gitea/gitea/pull/26214
(cherry picked from commit bb187d5f61)
(cherry picked from commit ce02ef9078)
(cherry picked from commit 6b75c40e25)
(cherry picked from commit 6bc8e9f573)
(cherry picked from commit 063f8afdf7)
(cherry picked from commit c5cc736b72)
(cherry picked from commit 7b1bb4bedc)
(cherry picked from commit 2a022dceb4)
(cherry picked from commit a946c142d2)
(cherry picked from commit 8a4ea0c7ab)
(cherry picked from commit 37bfb05b34)
(cherry picked from commit 12fbbb1754)
(cherry picked from commit abe9de2cd9)
(cherry picked from commit 84db578717)
(cherry picked from commit 1beab7af46)
(cherry picked from commit 8779d505f2)
(cherry picked from commit 21a788f2eb)
(cherry picked from commit a7cc6d1687)
(cherry picked from commit 25f840379f)
(cherry picked from commit 0db65f1122)
(cherry picked from commit 0e08cbc854)
(cherry picked from commit 57de51db8d)
(cherry picked from commit 5ab3dcabb1)
(cherry picked from commit 7b5ad0c13b)
(cherry picked from commit adf449a115)
(cherry picked from commit 77aa18227e)
(cherry picked from commit f9d75d4705)
(cherry picked from commit 64f76f4ab2)
(cherry picked from commit 5d02454155)
[CI] Forgejo Actions workflows
(cherry picked from commit 3ff59b5379)
(cherry picked from commit 8af826a6f7)
(cherry picked from commit d7c09d9cc8)
[CI] use the docker label instead of ubuntu-latest
(cherry picked from commit b6a6470db6)
[CI] all tests need compliance before proceeding
(cherry picked from commit b35c496f2c)
(cherry picked from commit 36a4148a8e)
(cherry picked from commit 7ffcffa653)
(cherry picked from commit 8a246d296e)
(cherry picked from commit dd0b6e1826)
[CI] Forgejo Actions based release process (squash) MySQL optimization
Refs: https://codeberg.org/forgejo/forgejo/issues/976
(cherry picked from commit b4b8c489e6)
(cherry picked from commit 1e861db4af)
(cherry picked from commit a6c0e00330)
(cherry picked from commit f97b336465)
(cherry picked from commit 6d65d5f0d6f798556a8f9e547896be03a5ee2f87)
(cherry picked from commit 79bfbadbed)
(cherry picked from commit e86c40a34a)
(cherry picked from commit da0c454adb)
(cherry picked from commit b49d892cda)
[CI] enable minio tests
(cherry picked from commit 4d8f438031)
(cherry picked from commit c4eeb0a61e)
[CI] Forgejo Actions based CI for PR & branches (squash) cleanup
(cherry picked from commit 80eb20e842)
(cherry picked from commit d2ff589858)
(cherry picked from commit f6eedecb67)
(cherry picked from commit cf458091e2)
(cherry picked from commit ddd322cb2d)
(cherry picked from commit f0f5729b64)
Conflicts:
.github/workflows/pull-db-tests.yml
https://codeberg.org/forgejo/forgejo/pulls/1573
(cherry picked from commit bb347aedd4)
(cherry picked from commit 0f5ebe8c3e)
(cherry picked from commit 9101cb4715)
(cherry picked from commit ab118fe4ef)
[CI] Forgejo Actions based CI for PR & branches (squash) use node:20-bookworm
No longer use the custom test-env image, it is unecessary technical
debt.
Also upgrade to bitnami/minio:2023.8.31 to align with what Gitea tests
(cherry picked from commit d9b77fd273)
(cherry picked from commit ef8f099f9e)
(cherry picked from commit 6d3c675d20)
(cherry picked from commit fc00ff45f2)
(cherry picked from commit b7f02b9846)
(cherry picked from commit d23d86e567)
(cherry picked from commit e68e65460c)
(cherry picked from commit 96f93a1fdb)
(cherry picked from commit 2ff3080018)
(cherry picked from commit 8ea00bcff4)
(cherry picked from commit 85be961970)
(cherry picked from commit 387b279fc3)
(cherry picked from commit 219be7aa85)
(cherry picked from commit 929b75ee39)
(cherry picked from commit dd42a30d6a)
(cherry picked from commit 8b11cab677)
(cherry picked from commit be59270696)
(cherry picked from commit e068f8b191)
(cherry picked from commit 7855bb0c60)
(cherry picked from commit 45c4c8f443)
(cherry picked from commit 89520d67ff)
(cherry picked from commit 15eeb417a4)
(cherry picked from commit 6db53a2643)
(cherry picked from commit 2f689b321f)
(cherry picked from commit 04dc478314c3b4927cca78c354ca46ee217f035a)
(cherry picked from commit a554624f40)
(cherry picked from commit abca05f0d1)
(cherry picked from commit dc13e7eb22)
(cherry picked from commit a161c5740e)
(cherry picked from commit 06d33e2773)
(cherry picked from commit f536275161)
(cherry picked from commit 84ac6f314a)
(cherry picked from commit 1e8126edfc)
(cherry picked from commit 0287ac3416)
(cherry picked from commit 3e5fca2aae)
(cherry picked from commit 03b220bfeb)
(cherry picked from commit 1d033f4aaf)
(cherry picked from commit 2ee9e3e9a3)
(cherry picked from commit d28c284993)
(cherry picked from commit 239df83859)
(cherry picked from commit 96ae0c2e5d)
(cherry picked from commit 49aef71b32)
(cherry picked from commit 38b56d108d)
(cherry picked from commit 30f8d9ec3a)
(cherry picked from commit d531861850)
(cherry picked from commit a75707deaa)
(cherry picked from commit b1c73918b2)
(cherry picked from commit 5391917091)
(cherry picked from commit a427f8dae5)
(cherry picked from commit 6ba6f62c7e)
(cherry picked from commit a1381d9146fba42cb97d72d38525fa3e721bfb03)
(cherry picked from commit 74714e0246)
(cherry picked from commit 7749dbfe66)
(cherry picked from commit 4379249711)
(cherry picked from commit a69f55bebf)
(cherry picked from commit 24dd5fbfdb)
(cherry picked from commit dda856d6b8)
(cherry picked from commit bc14f4fa97)
(cherry picked from commit 78fef4f137)
(cherry picked from commit 69e013cc51)
(cherry picked from commit f173c6a273)
(cherry picked from commit 92f9d02547)
(cherry picked from commit c99d51e665)
(cherry picked from commit aa0650fd2b)
(cherry picked from commit 0a8ef91302)
(cherry picked from commit 7b54fe01c2)
(cherry picked from commit 0e154f366f)
(cherry picked from commit 02d88ee16d)
(cherry picked from commit 411924e017)
(cherry picked from commit f4e9ca6db5)
(cherry picked from commit cd80126a23)
(cherry picked from commit da626702f9)
(cherry picked from commit 4b81d0bd04)
(cherry picked from commit 53ac260669)
(cherry picked from commit 984081f08d)
(cherry picked from commit 1c39bae3ec)
(cherry picked from commit e1bbfa3619)
(cherry picked from commit 91245ca917)
(cherry picked from commit 705d0558be)
(cherry picked from commit 9247594970)
(cherry picked from commit 9db1158a48)
(cherry picked from commit 3b36b77d87)
(cherry picked from commit 162fa1d8ae)
(cherry picked from commit d03d0afbb5)
(cherry picked from commit 7b8f92f787)
(cherry picked from commit 035abca969)
(cherry picked from commit a8fbf6bb56)
(cherry picked from commit 3be681d037b07880236cae1aa70245e5eb4d1497)
(cherry picked from commit 7e5d471c83)
(cherry picked from commit 323801d935)
(cherry picked from commit 3fdfe4bfea)
(cherry picked from commit 58a07421a4)
(cherry picked from commit dbb71a4c85)
(cherry picked from commit d442113520)
(cherry picked from commit d3329f01f8)
(cherry picked from commit 069a1d68b8)
(cherry picked from commit 14919e609a)
(cherry picked from commit 49b76be106)
(cherry picked from commit 0fe9f257d2)
(cherry picked from commit b583bebeab)
(cherry picked from commit 5c616e43a6)
(cherry picked from commit 854bcea905)
(cherry picked from commit c2acb181c5)
(cherry picked from commit 1cb07e71d1)
(cherry picked from commit 5d3f09e635)
(cherry picked from commit f8bf1c8d42)
(cherry picked from commit a471ed4576)
(cherry picked from commit 95c755f4e3)
(cherry picked from commit 1d8bc5215f)
(cherry picked from commit 45c1e7b8d0)
(cherry picked from commit 2eb4d93af7)
(cherry picked from commit 98dbce5e14)
(cherry picked from commit fbe2fb5861)
Resolves https://github.com/go-gitea/gitea/issues/28704
Example of an entry in the generated `APKINDEX` file:
```
C:Q1xCO3H9LTTEbhKt9G1alSC87I56c=
P:hello
V:2.12-r1
A:x86_64
T:The GNU Hello program produces a familiar, friendly greeting
U:https://www.gnu.org/software/hello/
L:GPL-3.0-or-later
S:15403
I:36864
o:hello
m:
t:1705934118
D:so:libc.musl-x86_64.so.1
p:cmd:hello=2.12-r1
i:foobar=1.0 !baz
k:42
```
the `i:` and `k:` entries are new.
---------
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Fixes#28660
Fixes an admin api bug related to `user.LoginSource`
Fixed `/user/emails` response not identical to GitHub api
This PR unifies the user update methods. The goal is to keep the logic
only at one place (having audit logs in mind). For example, do the
password checks only in one method not everywhere a password is updated.
After that PR is merged, the user creation should be next.
In #28691, schedule plans will be deleted when a repo's actions unit is
disabled. But when the unit is enabled, the schedule plans won't be
created again.
This PR fixes the bug. The schedule plans will be created again when the
actions unit is re-enabled
Renames it to `ENABLED` to be consistent with other settings and
deprecates it.
I believe this change is necessary because other setting groups such as
`attachment`, `cors`, `mailer`, etc. have an `ENABLED` setting, but
`oauth2` is the only one with an `ENABLE` setting, which could cause
confusion for users.
This is no longer a breaking change because `ENABLE` has been set as
deprecated and as an alias to `ENABLED`.
## Purpose
This is a refactor toward building an abstraction over managing git
repositories.
Afterwards, it does not matter anymore if they are stored on the local
disk or somewhere remote.
## What this PR changes
We used `git.OpenRepository` everywhere previously.
Now, we should split them into two distinct functions:
Firstly, there are temporary repositories which do not change:
```go
git.OpenRepository(ctx, diskPath)
```
Gitea managed repositories having a record in the database in the
`repository` table are moved into the new package `gitrepo`:
```go
gitrepo.OpenRepository(ctx, repo_model.Repo)
```
Why is `repo_model.Repository` the second parameter instead of file
path?
Because then we can easily adapt our repository storage strategy.
The repositories can be stored locally, however, they could just as well
be stored on a remote server.
## Further changes in other PRs
- A Git Command wrapper on package `gitrepo` could be created. i.e.
`NewCommand(ctx, repo_model.Repository, commands...)`. `git.RunOpts{Dir:
repo.RepoPath()}`, the directory should be empty before invoking this
method and it can be filled in the function only. #28940
- Remove the `RepoPath()`/`WikiPath()` functions to reduce the
possibility of mistakes.
---------
Co-authored-by: delvh <dev.lh@web.de>
The `ToUTF8*` functions were stripping BOM, while BOM is actually valid
in UTF8, so the stripping must be optional depending on use case. This
does:
- Add a options struct to all `ToUTF8*` functions, that by default will
strip BOM to preserve existing behaviour
- Remove `ToUTF8` function, it was dead code
- Rename `ToUTF8WithErr` to `ToUTF8`
- Preserve BOM in Monaco Editor
- Remove a unnecessary newline in the textarea value. Browsers did
ignore it, it seems but it's better not to rely on this behaviour.
Fixes: https://github.com/go-gitea/gitea/issues/28743
Related: https://github.com/go-gitea/gitea/issues/6716 which seems to
have once introduced a mechanism that strips and re-adds the BOM, but
from what I can tell, this mechanism was removed at some point after
that PR.
Fix#22066
# Purpose
This PR fix the releases will be deleted when mirror repository sync the
tags.
# The problem
In the previous implementation of #19125. All releases record in
databases of one mirror repository will be deleted before sync.
Ref:
https://github.com/go-gitea/gitea/pull/19125/files#diff-2aa04998a791c30e5a02b49a97c07fcd93d50e8b31640ce2ddb1afeebf605d02R481
# The Pros
This PR introduced a new method which will load all releases from
databases and all tags on git data into memory. And detect which tags
needs to be inserted, which tags need to be updated or deleted. Only
tags releases(IsTag=true) which are not included in git data will be
deleted, only tags which sha1 changed will be updated. So it will not
delete any real releases include drafts.
# The Cons
The drawback is the memory usage will be higher than before if there are
many tags on this repository. This PR defined a special release struct
to reduce columns loaded from database to memory.
This should fix https://github.com/go-gitea/gitea/issues/28927
Technically older versions of Git would support this flag as well, but
per https://github.com/go-gitea/gitea/pull/28466 that's the version
where using it (object-format=sha256) left "experimental" state.
`sha1` is (currently) the default, so older clients should be unaffected
in either case.
Signed-off-by: jolheiser <john.olheiser@gmail.com>
When LFS hooks are present in gitea-repositories, operations like git
push for creating a pull request fail. These repositories are not meant
to include LFS files or git push them, that is handled separately. And
so they should not have LFS hooks.
Installing git-lfs on some systems (like Debian Linux) will
automatically set up /etc/gitconfig to create LFS hooks in repositories.
For most git commands in Gitea this is not a problem, either because
they run on a temporary clone or the git command does not create LFS
hooks.
But one case where this happens is git archive for creating repository
archives. To fix that, add a GIT_CONFIG_NOSYSTEM=1 to disable using the
system configuration for that command.
According to a comment, GIT_CONFIG_NOSYSTEM is not used for all git
commands because the system configuration can be intentionally set up
for Gitea to use.
Resolves#19810, #21148
Sometimes you need to work on a feature which depends on another (unmerged) feature.
In this case, you may create a PR based on that feature instead of the main branch.
Currently, such PRs will be closed without the possibility to reopen in case the parent feature is merged and its branch is deleted.
Automatic target branch change make life a lot easier in such cases.
Github and Bitbucket behave in such way.
Example:
$PR_1$: main <- feature1
$PR_2$: feature1 <- feature2
Currently, merging $PR_1$ and deleting its branch leads to $PR_2$ being closed without the possibility to reopen.
This is both annoying and loses the review history when you open a new PR.
With this change, $PR_2$ will change its target branch to main ($PR_2$: main <- feature2) after $PR_1$ has been merged and its branch has been deleted.
This behavior is enabled by default but can be disabled.
For security reasons, this target branch change will not be executed when merging PRs targeting another repo.
Fixes#27062Fixes#18408
---------
Co-authored-by: Denys Konovalov <kontakt@denyskon.de>
Co-authored-by: delvh <dev.lh@web.de>
Fixes#26548
This PR refactors the rendering of markup links. The old code uses
`strings.Replace` to change some urls while the new code uses more
context to decide which link should be generated.
The added tests should ensure the same output for the old and new
behaviour (besides the bug).
We may need to refactor the rendering a bit more to make it clear how
the different helper methods render the input string. There are lots of
options (resolve links / images / mentions / git hashes / emojis / ...)
but you don't really know what helper uses which options. For example,
we currently support images in the user description which should not be
allowed I think:
<details>
<summary>Profile</summary>
https://try.gitea.io/KN4CK3R
</details>
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Fixes#27114.
* In Gitea 1.12 (#9532), a "dismiss stale approvals" branch protection
setting was introduced, for ignoring stale reviews when verifying the
approval count of a pull request.
* In Gitea 1.14 (#12674), the "dismiss review" feature was added.
* This caused confusion with users (#25858), as "dismiss" now means 2
different things.
* In Gitea 1.20 (#25882), the behavior of the "dismiss stale approvals"
branch protection was modified to actually dismiss the stale review.
For some users this new behavior of dismissing the stale reviews is not
desirable.
So this PR reintroduces the old behavior as a new "ignore stale
approvals" branch protection setting.
---------
Co-authored-by: delvh <dev.lh@web.de>
Fix#28157
This PR fix the possible bugs about actions schedule.
## The Changes
- Move `UpdateRepositoryUnit` and `SetRepoDefaultBranch` from models to
service layer
- Remove schedules plan from database and cancel waiting & running
schedules tasks in this repository when actions unit has been disabled
or global disabled.
- Remove schedules plan from database and cancel waiting & running
schedules tasks in this repository when default branch changed.
Mainly for MySQL/MSSQL.
It is important for Gitea to use case-sensitive database charset
collation. If the database is using a case-insensitive collation, Gitea
will show startup error/warning messages, and show the errors/warnings
on the admin panel's Self-Check page.
Make `gitea doctor convert` work for MySQL to convert the collations of
database & tables & columns.
* Fix#28131
## ⚠️ BREAKING ⚠️
It is not quite breaking, but it's highly recommended to convert the
database&table&column to a consistent and case-sensitive collation.
In #26365 issue references were disabled entirely for documents,
intending to match GitHub behavior. However cross-references do appear
to work in documents on GitHub.
This is useful for example to write release notes in a markdown document
and reference issues. While the simpler syntax may create links when not
intended, hopefully the cross-reference syntax is unique enough to avoid
it.
The CORS code has been unmaintained for long time, and the behavior is
not correct.
This PR tries to improve it. The key point is written as comment in
code. And add more tests.
Fix#28515Fix#27642Fix#17098
Fix#28526, regression of
* #26365
(although the author of #26365 has recent activities, but there is no
response for the regression, so I proposed this quick fix and keep the
fix simple to make it easier to backport to 1.21)
Nowadays, cache will be used on almost everywhere of Gitea and it cannot
be disabled, otherwise some features will become unaviable.
Then I think we can just remove the option for cache enable. That means
cache cannot be disabled.
But of course, we can still use cache configuration to set how should
Gitea use the cache.
The 4 functions are duplicated, especially as interface methods. I think
we just need to keep `MustID` the only one and remove other 3.
```
MustID(b []byte) ObjectID
MustIDFromString(s string) ObjectID
NewID(b []byte) (ObjectID, error)
NewIDFromString(s string) (ObjectID, error)
```
Introduced the new interfrace method `ComputeHash` which will replace
the interface `HasherInterface`. Now we don't need to keep two
interfaces.
Reintroduced `git.NewIDFromString` and `git.MustIDFromString`. The new
function will detect the hash length to decide which objectformat of it.
If it's 40, then it's SHA1. If it's 64, then it's SHA256. This will be
right if the commitID is a full one. So the parameter should be always a
full commit id.
@AdamMajer Please review.
- Modify the `Password` field in `CreateUserOption` struct to remove the
`Required` tag
- Update the `v1_json.tmpl` template to include the `email` field and
remove the `password` field
---------
Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
Update golang.org/x/crypto for CVE-2023-48795 and update other packages.
`go-git` is not updated because it needs time to figure out why some
tests fail.
- If a topic has zero repository count, it means that none of the
repositories are using that topic, that would make them 'useless' to
keep. One caveat is that if that topic is going to be used in the
future, it will be added again to the database, but simply with a new
ID.
Refs: https://codeberg.org/forgejo/forgejo/pulls/1964
Co-authored-by: Gusted <postmaster@gusted.xyz>
- Remove `ObjectFormatID`
- Remove function `ObjectFormatFromID`.
- Use `Sha1ObjectFormat` directly but not a pointer because it's an
empty struct.
- Store `ObjectFormatName` in `repository` struct
- When a repository is orphaned and has objects stored in any of the
storages such as repository avatar or attachments the delete function
would error, because the storage module wasn't initalized.
- Add code to initialize the storage module.
Refs: https://codeberg.org/forgejo/forgejo/pulls/1954
Co-authored-by: Gusted <postmaster@gusted.xyz>
Refactor Hash interfaces and centralize hash function. This will allow
easier introduction of different hash function later on.
This forms the "no-op" part of the SHA256 enablement patch.
## Changes
- Add deprecation warning to `Token` and `AccessToken` authentication
methods in swagger.
- Add deprecation warning header to API response. Example:
```
HTTP/1.1 200 OK
...
Warning: token and access_token API authentication is deprecated
...
```
- Add setting `DISABLE_QUERY_AUTH_TOKEN` to reject query string auth
tokens entirely. Default is `false`
## Next steps
- `DISABLE_QUERY_AUTH_TOKEN` should be true in a subsequent release and
the methods should be removed in swagger
- `DISABLE_QUERY_AUTH_TOKEN` should be removed and the implementation of
the auth methods in question should be removed
## Open questions
- Should there be further changes to the swagger documentation?
Deprecation is not yet supported for security definitions (coming in
[OpenAPI Spec version
3.2.0](https://github.com/OAI/OpenAPI-Specification/issues/2506))
- Should the API router logger sanitize urls that use `token` or
`access_token`? (This is obviously an insufficient solution on its own)
---------
Co-authored-by: delvh <dev.lh@web.de>
1. Do not sort the "checks" slice again and again when "Register", it
just wastes CPU when the Gitea instance runs
2. If a check doesn't exist, tell the end user
3. Add some tests
The function `GetByBean` has an obvious defect that when the fields are
empty values, it will be ignored. Then users will get a wrong result
which is possibly used to make a security problem.
To avoid the possibility, this PR removed function `GetByBean` and all
references.
And some new generic functions have been introduced to be used.
The recommand usage like below.
```go
// if query an object according id
obj, err := db.GetByID[Object](ctx, id)
// query with other conditions
obj, err := db.Get[Object](ctx, builder.Eq{"a": a, "b":b})
```
It will fix#28268 .
<img width="1313" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/cb1e07d5-7a12-4691-a054-8278ba255bfc">
<img width="1318" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/4fd60820-97f1-4c2c-a233-d3671a5039e9">
## ⚠️ BREAKING ⚠️
But need to give up some features:
<img width="1312" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/281c0d51-0e7d-473f-bbed-216e2f645610">
However, such abandonment may fix#28055 .
## Backgroud
When the user switches the dashboard context to an org, it means they
want to search issues in the repos that belong to the org. However, when
they switch to themselves, it means all repos they can access because
they may have created an issue in a public repo that they don't own.
<img width="286" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/182dcd5b-1c20-4725-93af-96e8dfae5b97">
It's a confusing design. Think about this: What does "In your
repositories" mean when the user switches to an org? Repos belong to the
user or the org?
Whatever, it has been broken by #26012 and its following PRs. After the
PR, it searches for issues in repos that the dashboard context user owns
or has been explicitly granted access to, so it causes #28268.
## How to fix it
It's not really difficult to fix it. Just extend the repo scope to
search issues when the dashboard context user is the doer. Since the
user may create issues or be mentioned in any public repo, we can just
set `AllPublic` to true, which is already supported by indexers. The DB
condition will also support it in this PR.
But the real difficulty is how to count the search results grouped by
repos. It's something like "search issues with this keyword and those
filters, and return the total number and the top results. **Then, group
all of them by repo and return the counts of each group.**"
<img width="314" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/5206eb20-f8f5-49b9-b45a-1be2fcf679f4">
Before #26012, it was being done in the DB, but it caused the results to
be incomplete (see the description of #26012).
And to keep this, #26012 implement it in an inefficient way, just count
the issues by repo one by one, so it cannot work when `AllPublic` is
true because it's almost impossible to do this for all public repos.
1bfcdeef4c/modules/indexer/issues/indexer.go (L318-L338)
## Give up unnecessary features
We may can resovle `TODO: use "group by" of the indexer engines to
implement it`, I'm sure it can be done with Elasticsearch, but IIRC,
Bleve and Meilisearch don't support "group by".
And the real question is, does it worth it? Why should we need to know
the counts grouped by repos?
Let me show you my search dashboard on gitea.com.
<img width="1304" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/2bca2d46-6c71-4de1-94cb-0c9af27c62ff">
I never think the long repo list helps anything.
And if we agree to abandon it, things will be much easier. That is this
PR.
## TODO
I know it's important to filter by repos when searching issues. However,
it shouldn't be the way we have it now. It could be implemented like
this.
<img width="1316" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/99ee5f21-cbb5-4dfe-914d-cb796cb79fbe">
The indexers support it well now, but it requires some frontend work,
which I'm not good at. So, I think someone could help do that in another
PR and merge this one to fix the bug first.
Or please block this PR and help to complete it.
Finally, "Switch dashboard context" is also a design that needs
improvement. In my opinion, it can be accomplished by adding filtering
conditions instead of "switching".
The summary string ends up in the database, and (at least) MySQL &
PostgreSQL require valid UTF8 strings.
Fixes#28178
Co-authored-by: Darrin Smart <darrin@filmlight.ltd.uk>
Previously only the first term had to be matched. That default
Meilisearch behavior makes sense for e.g. some kind of autocomplete to
find and select a single result. But for filtering issues it means you
can't narrow down results by adding more terms.
This is also more consistent with other indexers and GitHub.
---
Reference:
https://www.meilisearch.com/docs/reference/api/search#matching-strategy
The git command may operate the git directory (add/remove) files in any
time.
So when the code iterates the directory, some files may disappear during
the "walk". All "IsNotExist" errors should be ignored.
Fix#26765
gitea doctor failed at checking and fixing 'delete-orphaned-repos',
because table name 'user' needs quoting to be correctly recognized by at
least PostgreSQL.
fixes#28199
- Currently the repository description uses the same sanitizer as a
normal markdown document. This means that element such as heading and
images are allowed and can be abused.
- Create a minimal restricted sanitizer for the repository description,
which only allows what the postprocessor currently allows, which are
links and emojis.
- Added unit testing.
- Resolves https://codeberg.org/forgejo/forgejo/issues/1202
- Resolves https://codeberg.org/Codeberg/Community/issues/1122
(cherry picked from commit 631c87cc23)
Co-authored-by: Gusted <postmaster@gusted.xyz>
Fix#25473
Although there was `m.Post("/login/oauth/access_token", CorsHandler()...`,
it never really worked, because it still lacks the "OPTIONS" handler.
Fixes#27819
We have support for two factor logins with the normal web login and with
basic auth. For basic auth the two factor check was implemented at three
different places and you need to know that this check is necessary. This
PR moves the check into the basic auth itself.
- On user deletion, delete action runners that the user has created.
- Add a database consistency check to remove action runners that have
nonexistent belonging owner.
- Resolves https://codeberg.org/forgejo/forgejo/issues/1720
(cherry picked from commit 009ca7223d)
Co-authored-by: Gusted <postmaster@gusted.xyz>
This patchset changes the connection string builder to use net.URL and
the host/port parser to use the stdlib function for splitting host from
port. It also adds a footnote about a potentially required portnumber
for postgres UNIX sockets.
Fixes: #24552
This PR adds a prefix path for all minio storage and override base path
will override the path.
The previous behavior is undefined officially, so it will be marked as
breaking.
After many refactoring PRs for the "locale" and "template context
function", now the ".locale" is not needed for web templates any more.
This PR does a clean up for:
1. Remove `ctx.Data["locale"]` for web context.
2. Use `ctx.Locale` in `500.tmpl`, for consistency.
3. Add a test check for `500 page` locale usage.
4. Remove the `Str2html` and `DotEscape` from mail template context
data, they are copy&paste errors introduced by #19169 and #16200 . These
functions are template functions (provided by the common renderer), but
not template data variables.
5. Make email `SendAsync` function mockable (I was planning to add more
tests but it would make this PR much too complex, so the tests could be
done in another PR)
See https://github.com/go-gitea/gitea/pull/27718#issuecomment-1773743014
. Add a test to ensure its behavior.
Why this test uses `ProjectBoardID=0`? Because in `SearchOptions`,
`ProjectBoardID=0` means what it is. But in `IssueOptions`,
`ProjectBoardID=0` means there is no condition, and
`ProjectBoardID=db.NoConditionID` means the board ID = 0.
It's really confusing. Probably it's better to separate the db search
engine and the other issue search code. It's really two different
systems. As far as I can see, `IssueOptions` is not necessary for most
of the code, which has very simple issue search conditions.
1. remove unused function `MoveIssueAcrossProjectBoards`
2. extract the project board condition into a function
3. use db.NoCondition instead of -1. (BTW, the usage of db.NoCondition
is too confusing. Is there any way to avoid that?)
4. remove the unnecessary comment since the ctx refactor is completed.
5. Change `b.ID != 0` to `b.ID > 0`. It's more intuitive but I think
they're the same since board ID can't be negative.
When `webhook.PROXY_URL` has been set, the old code will check if the
proxy host is in `ALLOWED_HOST_LIST` or reject requests through the
proxy. It requires users to add the proxy host to `ALLOWED_HOST_LIST`.
However, it actually allows all requests to any port on the host, when
the proxy host is probably an internal address.
But things may be even worse. `ALLOWED_HOST_LIST` doesn't really work
when requests are sent to the allowed proxy, and the proxy could forward
them to any hosts.
This PR fixes it by:
- If the proxy has been set, always allow connectioins to the host and
port.
- Check `ALLOWED_HOST_LIST` before forwarding.
Closes#27455
> The mechanism responsible for long-term authentication (the 'remember
me' cookie) uses a weak construction technique. It will hash the user's
hashed password and the rands value; it will then call the secure cookie
code, which will encrypt the user's name with the computed hash. If one
were able to dump the database, they could extract those two values to
rebuild that cookie and impersonate a user. That vulnerability exists
from the date the dump was obtained until a user changed their password.
>
> To fix this security issue, the cookie could be created and verified
using a different technique such as the one explained at
https://paragonie.com/blog/2015/04/secure-authentication-php-with-long-term-persistence#secure-remember-me-cookies.
The PR removes the now obsolete setting `COOKIE_USERNAME`.
assert.Fail() will continue to execute the code while assert.FailNow()
not. I thought those uses of assert.Fail() should exit immediately.
PS: perhaps it's a good idea to use
[require](https://pkg.go.dev/github.com/stretchr/testify/require)
somewhere because the assert package's default behavior does not exit
when an error occurs, which makes it difficult to find the root error
reason.
Fix #27541
The INI package has a quirk: by default, the keys are inherited.
When maintaining the keys, the newly added sub key should not be
affected by the parent key.
Part of https://github.com/go-gitea/gitea/issues/27097:
- `gitea` theme is renamed to `gitea-light`
- `arc-green` theme is renamed to `gitea-dark`
- `auto` theme is renamed to `gitea-auto`
I put both themes in separate CSS files, removing all colors from the
base CSS. Existing users will be migrated to the new theme names. The
dark theme recolor will follow in a separate PR.
## ⚠️ BREAKING ⚠️
1. If there are existing custom themes with the names `gitea-light` or
`gitea-dark`, rename them before this upgrade and update the `theme`
column in the `user` table for each affected user.
2. The theme in `<html>` has moved from `class="theme-name"` to
`data-theme="name"`, existing customizations that depend on should be
updated.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
This pull request is a minor code cleanup.
From the Go specification (https://go.dev/ref/spec#For_range):
> "1. For a nil slice, the number of iterations is 0."
> "3. If the map is nil, the number of iterations is 0."
`len` returns 0 if the slice or map is nil
(https://pkg.go.dev/builtin#len). Therefore, checking `len(v) > 0`
before a loop is unnecessary.
---
At the time of writing this pull request, there wasn't a lint rule that
catches these issues. The closest I could find is
https://staticcheck.dev/docs/checks/#S103
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
This PR reduces the complexity of the system setting system.
It only needs one line to introduce a new option, and the option can be
used anywhere out-of-box.
It is still high-performant (and more performant) because the config
values are cached in the config system.
With this PR we added the possibility to configure the Actions timeouts
values for killing tasks/jobs.
Particularly this enhancement is closely related to the `act_runner`
configuration reported below:
```
# The timeout for a job to be finished.
# Please note that the Gitea instance also has a timeout (3h by default) for the job.
# So the job could be stopped by the Gitea instance if it's timeout is shorter than this.
timeout: 3h
```
---
Setting the corresponding key in the INI configuration file, it is
possible to let jobs run for more than 3 hours.
Signed-off-by: Francesco Antognazza <francesco.antognazza@gmail.com>
Part of #27065
This PR touches functions used in templates. As templates are not static
typed, errors are harder to find, but I hope I catch it all. I think
some tests from other persons do not hurt.
Follow #23465 and #25624
This PR introduces the following improvements:
- We do not need to call `GetTags` to get tags because tags have been
loaded by `RepoAssignment`
ef90fdbd1d/modules/context/repo.go (L663-L668)
- Similarly, the number of tags and releases also have been loaded by
`RepoAssignment`, so the related code has been removed from the
handlers. The query condition of `GetReleaseCountByRepoID` in
`RepoAssignment` has been changed to include draft releases.
ef90fdbd1d/modules/context/repo.go (L538-L551)
- `releasesOrTags` function has been removed. The code for rendering
releases list and tags list moved to `Releases` and `TagList`
respectively.
This PR removed `unittest.MainTest` the second parameter
`TestOptions.GiteaRoot`. Now it detects the root directory by current
working directory.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
I noticed, that the push mirrors endpoint, is the only endpoint which
returns the times in long format rather than as time.Time().
I think the behavior should be consistent across the project.
----
## ⚠️ BREAKING ⚠️
This PR changes the time format used in API responses for all
push_mirror endpoints which return a push mirror.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
- Refactor lfs request code
- The original code uses `performRequest` function to create the
request, uses a callback to modify the request, and then send the
request.
- Now it's replaced with `createRequest` that only creates request and
`performRequest` that only sends the request.
- Reuse `createRequest` and `performRequest` in `http_client.go` and
`transferadapter.go`
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Closes#26329
This PR adds the ability to ignore revisions specified in the
`.git-blame-ignore-revs` file in the root of the repository.
The banner is displayed in this case. I intentionally did not add a UI
way to bypass the ignore file (same behaviour as Github) but you can add
`?bypass-blame-ignore=true` to the url manually.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This PR adds a new field `RemoteAddress` to both mirror types which
contains the sanitized remote address for easier (database) access to
that information. Will be used in the audit PR if merged.
As described in the title.
Some points:
1. Why need those tests?
Because `buildIssueOverview` is not well tested, there are several
continuous bugs in the issue overview webpage.
2. Why in indexer_test.go?
It's hard to put those tests in `./modules/indexer/issue/db/db_test.go`
because those tests need 'real' data in db mocked by fixtures instead of
random data in `./modules/indexer/issue/internal/tests`. When using
'real' data(`unittest.PrepareTestDatabase`), `InitIssueIndexer` and the
package `init()` function of `indexer` are required to init indexer.
3. Why only db?
The other three indexer engines are well tested by random data and it's
okay to also test them with 'real' data in db mocked by fixtures. Any
follow-up PR is welcome.
4. Those tests are really basic, any more complicated tests are welcome.
5. I think it's also necessary to add tests in `TestAPISearchIssues`
in`api_test_issue.go` and `TestIssues` in `home_test.go`
Part of #27065
This reduces the usage of `db.DefaultContext`. I think I've got enough
files for the first PR. When this is merged, I will continue working on
this.
Considering how many files this PR affect, I hope it won't take to long
to merge, so I don't end up in the merge conflict hell.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Since the issue indexer has been refactored, the issue overview webpage
is built by the `buildIssueOverview` function and underlying
`indexer.Search` function and `GetIssueStats` instead of
`GetUserIssueStats`. So the function is no longer used.
I moved the relevant tests to `indexer_test.go` and since the search
option changed from `IssueOptions` to `SearchOptions`, most of the tests
are useless now.
We need more tests about the db indexer because those tests are highly
connected with the issue overview webpage and now this page has several
bugs.
Any advice about those test cases is appreciated.
---------
Co-authored-by: CaiCandong <50507092+CaiCandong@users.noreply.github.com>
- In org mode you can specify an description for media via the following
syntax `[[description][media link]]`. The description is then used as
title or alt.
- This patch fixes the rendering of the description by seperating the
description and non-description cases and using `org.String()`.
- Added unit tests.
- Inspired by
6eb20dbda9/org/html_writer.go (L406-L427)
- Resolves https://codeberg.org/Codeberg/Community/issues/848
(cherry picked from commit 8b8aab8311)
Co-authored-by: Gusted <postmaster@gusted.xyz>
Fix#26723
Add `ChangeDefaultBranch` to the `notifier` interface and implement it
in `indexerNotifier`. So when changing the default branch,
`indexerNotifier` sends a message to the `indexer queue` to update the
index.
---------
Co-authored-by: techknowlogick <matti@mdranta.net>
Should BucketExists (HeadBucket) fail because of an error related to
the connection rather than the existence of the bucket, no information
is available and the admin is left guessing.
https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadBucket.html
> This action is useful to determine if a bucket exists and you have
> permission to access it. The action returns a 200 OK if the bucket
> exists and you have permission to access it.
>
> If the bucket does not exist or you do not have permission to access
> it, the HEAD request returns a generic 400 Bad Request, 403
> Forbidden or 404 Not Found code. A message body is not included, so
> you cannot determine the exception beyond these error codes.
GetBucketVersioning is used instead and exclusively dedicated to
asserting if using the connection does not return a BadRequest.
If it does the NewMinioStorage logs an error and returns. Otherwise
it keeps going knowing that BucketExists is not going to fail for
reasons unrelated to the existence of the bucket and the permissions
to access it.
(cherry picked from commit d1df4b3bc62e5e61893a923f1c4b58f084eb03af)
Refs: https://codeberg.org/forgejo/forgejo/issues/1338
Unfortunately, when a system setting hasn't been stored in the database,
it cannot be cached.
Meanwhile, this PR also uses context cache for push email avatar display
which should avoid to read user table via email address again and again.
According to my local test, this should reduce dashboard elapsed time
from 150ms -> 80ms .
If the AppURL(ROOT_URL) is an HTTPS URL, then the COOKIE_SECURE's
default value should be true.
And, if a user visits an "http" site with "https" AppURL, they won't be
able to login, and they should have been warned. The only problem is
that the "language" can't be set either in such case, while I think it
is not a serious problem, and it could be fixed easily if needed.
A set of terminology, along with a broader description, can help more
people engage with the Gitea queue system, providing insights and
ensuring its correct use.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This feature was removed by #22219 to avoid possible CSRF attack.
This PR takes reverseproxy auth for API back but with default disabled.
To prevent possbile CSRF attack, the responsibility will be the
reverseproxy but not Gitea itself.
For those want to enable this `ENABLE_REVERSE_PROXY_AUTHENTICATION_API`,
they should know what they are doing.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Currently, Artifact does not have an expiration and automatic cleanup
mechanism, and this feature needs to be added. It contains the following
key points:
- [x] add global artifact retention days option in config file. Default
value is 90 days.
- [x] add cron task to clean up expired artifacts. It should run once a
day.
- [x] support custom retention period from `retention-days: 5` in
`upload-artifact@v3`.
- [x] artifacts link in actions view should be non-clickable text when
expired.
1. The `og:description` should be "a one to two sentence description of
your object"
* It shouldn't output all the user inputted content -- it would be
pretty huge.
* Maybe it only needs at most 300 bytes.
2. Do not render commit message as HTML
- While doing some sanity checks over OpenSSH's code for how they handle
certificates authentication. I stumbled on an condition that checks the
certificate type is really an user certificate on the server-side
authentication. This checks seems to be a formality and just for the
sake of good domain seperation, because an user and host certificate
don't differ in their generation, verification or flags that can be
included.
- Add this check to the builtin SSH server to stay close to the
unwritten SSH specification.
- This is an breaking change for setups where the builtin SSH server is
being used and for some reason host certificates were being used for
authentication.
-
(cherry picked from commit de35b141b7)
Refs: https://codeberg.org/forgejo/forgejo/pulls/1172
## ⚠️ BREAKING ⚠️
Like OpenSSH, the built-in SSH server will now only accept SSH user
certificates, not server certificates.
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-authored-by: Giteabot <teabot@gitea.io>
1. The old `prepareQueryArg` did double-unescaping of form value.
2. By the way, remove the unnecessary `ctx.Flash = ...` in
`MockContext`.
Co-authored-by: Giteabot <teabot@gitea.io>
Just like `models/unittest`, the testing helper functions should be in a
separate package: `contexttest`
And complete the TODO:
> // TODO: move this function to other packages, because it depends on
"models" package
Backtick syntax now works in repo description too. Also, I replaced the
CSS for this was a new single class, making it more flexible and not
dependent on a parent. Also, very slightly reduced font size from 16.8px
to 16px.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
From the Go specification:
> "1. For a nil slice, the number of iterations is 0."
https://go.dev/ref/spec#For_range
Therefore, an additional nil check for before the loop is unnecessary.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Related to: #8312#26491
In migration v109, we only added a new column `CanCreateOrgRepo` in Team
table, but not initial the value of it.
This may cause bug like #26491.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
According to the GitHub API Spec:
https://docs.github.com/en/rest/actions/secrets?apiVersion=2022-11-28#create-or-update-an-organization-secret
Merge the Create and Update secret into a single API.
- Remove the `CreateSecretOption` struct and replace it with
`CreateOrUpdateSecretOption` in `modules/structs/secret.go`
- Update the `CreateOrUpdateOrgSecret` function in
`routers/api/v1/org/action.go` to use `CreateOrUpdateSecretOption`
instead of `UpdateSecretOption`
- Remove the `CreateOrgSecret` function in
`routers/api/v1/org/action.go` and replace it with
`CreateOrUpdateOrgSecret`
- Update the Swagger documentation in
`routers/api/v1/swagger/options.go` and `templates/swagger/v1_json.tmpl`
to reflect the changes in the struct names and function names
Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
The web context (modules/context.Context) is quite complex, it's
difficult for the callers to initialize correctly.
This PR introduces a `NewWebContext` function, to make sure the web
context have the same behavior for different cases.
- Resolves https://codeberg.org/forgejo/forgejo/issues/580
- Return a `upload_field` to any release API response, which points to
the API URL for uploading new assets.
- Adds unit test.
- Adds integration testing to verify URL is returned correctly and that
upload endpoint actually works
---------
Co-authored-by: Gusted <postmaster@gusted.xyz>
Hi,
We'd like to add merge files files to GetCommitFileStatus fucntions so
API returns the list of all the files associated to a merged pull
request commit, like GitHub API does.
The list of affectedFiles for an API commit is fetched from toCommit()
function in routers/api/v1/repo/commits.go, and API was returning no
file in case of a pull request with no conflict, or just files
associated to the confict resolution, but NOT the full list of merged
files.
This would lead to situations where a CI polling a repo for changes
could miss some file changes due to API returning an empty / partial
list in case of such merged pull requests. (Hope this makes sense :) )
NOTE: I'd like to add a unittest in
integrations/api_repo_git_commits_test.go but failed to understand how
to add my own test bare repo so I can make a test on a merged pull
request commit to check for affectedFiles.
Is there a merged pull request in there that I could use maybe?
Could someone please direct me to the relevant ressources with
informations on how to do that please?
Thanks for your time,
Laurent.
---------
Co-authored-by: Thomas Desveaux <desveaux.thomas@gmail.com>
