Support for grouping RPMs using paths ()

The current rpm repository places all packages in the same repository,
and different systems (el7,f34) may hit packages that do not belong to
this distribution (  ) , which now supports grouping of rpm.

![图片](d1e1d99f-7799-4b2b-a19b-cb2a5c692914)

Fixes  .
Fixes  .

Refactor: [](https://github.com/go-gitea/gitea/pull/25866)
This commit is contained in:
Exploding Dragon 2024-01-12 11:16:05 +08:00 committed by GitHub
parent 7c2f093e85
commit ba4d0b8ffb
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
9 changed files with 192 additions and 101 deletions
docs/content/usage/packages
modules
packages/rpm
templates
routers/api/packages
services/packages/rpm
templates/package/content
tests/integration

View file

@ -27,17 +27,18 @@ The following examples use `dnf`.
To register the RPM registry add the url to the list of known apt sources: To register the RPM registry add the url to the list of known apt sources:
```shell ```shell
dnf config-manager --add-repo https://gitea.example.com/api/packages/{owner}/rpm.repo dnf config-manager --add-repo https://gitea.example.com/api/packages/{owner}/rpm/{group}.repo
``` ```
| Placeholder | Description | | Placeholder | Description |
| ----------- | ----------- | | ----------- |----------------------------------------------------|
| `owner` | The owner of the package. | | `owner` | The owner of the package. |
| `group` | Everything, e.g. `el7`, `rocky/el9` , `test/fc38`.|
If the registry is private, provide credentials in the url. You can use a password or a [personal access token](development/api-usage.md#authentication): If the registry is private, provide credentials in the url. You can use a password or a [personal access token](development/api-usage.md#authentication):
```shell ```shell
dnf config-manager --add-repo https://{username}:{your_password_or_token}@gitea.example.com/api/packages/{owner}/rpm.repo dnf config-manager --add-repo https://{username}:{your_password_or_token}@gitea.example.com/api/packages/{owner}/rpm/{group}.repo
``` ```
You have to add the credentials to the urls in the `rpm.repo` file in `/etc/yum.repos.d` too. You have to add the credentials to the urls in the `rpm.repo` file in `/etc/yum.repos.d` too.
@ -47,19 +48,20 @@ You have to add the credentials to the urls in the `rpm.repo` file in `/etc/yum.
To publish a RPM package (`*.rpm`), perform a HTTP PUT operation with the package content in the request body. To publish a RPM package (`*.rpm`), perform a HTTP PUT operation with the package content in the request body.
``` ```
PUT https://gitea.example.com/api/packages/{owner}/rpm/upload PUT https://gitea.example.com/api/packages/{owner}/rpm/{group}/upload
``` ```
| Parameter | Description | | Parameter | Description |
| --------- | ----------- | | --------- | ----------- |
| `owner` | The owner of the package. | | `owner` | The owner of the package. |
| `group` | Everything, e.g. `el7`, `rocky/el9` , `test/fc38`.|
Example request using HTTP Basic authentication: Example request using HTTP Basic authentication:
```shell ```shell
curl --user your_username:your_password_or_token \ curl --user your_username:your_password_or_token \
--upload-file path/to/file.rpm \ --upload-file path/to/file.rpm \
https://gitea.example.com/api/packages/testuser/rpm/upload https://gitea.example.com/api/packages/testuser/rpm/centos/el7/upload
``` ```
If you are using 2FA or OAuth use a [personal access token](development/api-usage.md#authentication) instead of the password. If you are using 2FA or OAuth use a [personal access token](development/api-usage.md#authentication) instead of the password.
@ -78,21 +80,22 @@ The server responds with the following HTTP Status codes.
To delete an RPM package perform a HTTP DELETE operation. This will delete the package version too if there is no file left. To delete an RPM package perform a HTTP DELETE operation. This will delete the package version too if there is no file left.
``` ```
DELETE https://gitea.example.com/api/packages/{owner}/rpm/{package_name}/{package_version}/{architecture} DELETE https://gitea.example.com/api/packages/{owner}/rpm/{group}/package/{package_name}/{package_version}/{architecture}
``` ```
| Parameter | Description | | Parameter | Description |
| ----------------- | ----------- | |-------------------|----------------------------|
| `owner` | The owner of the package. | | `owner` | The owner of the package. |
| `package_name` | The package name. | | `group` | The package group . |
| `package_version` | The package version. | | `package_name` | The package name. |
| `architecture` | The package architecture. | | `package_version` | The package version. |
| `architecture` | The package architecture. |
Example request using HTTP Basic authentication: Example request using HTTP Basic authentication:
```shell ```shell
curl --user your_username:your_token_or_password -X DELETE \ curl --user your_username:your_token_or_password -X DELETE \
https://gitea.example.com/api/packages/testuser/rpm/test-package/1.0.0/x86_64 https://gitea.example.com/api/packages/testuser/rpm/centos/el7/package/test-package/1.0.0/x86_64
``` ```
The server responds with the following HTTP Status codes. The server responds with the following HTTP Status codes.

View file

@ -27,17 +27,18 @@ menu:
要注册RPM注册表请将 URL 添加到已知 `apt` 源列表中: 要注册RPM注册表请将 URL 添加到已知 `apt` 源列表中:
```shell ```shell
dnf config-manager --add-repo https://gitea.example.com/api/packages/{owner}/rpm.repo dnf config-manager --add-repo https://gitea.example.com/api/packages/{owner}/rpm/{group}.repo
``` ```
| 占位符 | 描述 | | 占位符 | 描述 |
| ------- | -------------- | | ------- |--------------------------------------|
| `owner` | 软件包的所有者 | | `owner` | 软件包的所有者 |
| `group` | 任何名称,例如 `centos/7``el-7``fc38` |
如果注册表是私有的请在URL中提供凭据。您可以使用密码或[个人访问令牌](development/api-usage.md#通过-api-认证) 如果注册表是私有的请在URL中提供凭据。您可以使用密码或[个人访问令牌](development/api-usage.md#通过-api-认证)
```shell ```shell
dnf config-manager --add-repo https://{username}:{your_password_or_token}@gitea.example.com/api/packages/{owner}/rpm.repo dnf config-manager --add-repo https://{username}:{your_password_or_token}@gitea.example.com/api/packages/{owner}/rpm/{group}.repo
``` ```
您还必须将凭据添加到 `/etc/yum.repos.d` 中的 `rpm.repo` 文件中的URL中。 您还必须将凭据添加到 `/etc/yum.repos.d` 中的 `rpm.repo` 文件中的URL中。
@ -47,19 +48,20 @@ dnf config-manager --add-repo https://{username}:{your_password_or_token}@gitea.
要发布RPM软件包`*.rpm`),请执行带有软件包内容的 HTTP `PUT` 操作。 要发布RPM软件包`*.rpm`),请执行带有软件包内容的 HTTP `PUT` 操作。
``` ```
PUT https://gitea.example.com/api/packages/{owner}/rpm/upload PUT https://gitea.example.com/api/packages/{owner}/rpm/{group}/upload
``` ```
| 参数 | 描述 | | 参数 | 描述 |
| ------- | -------------- | | ------- |--------------|
| `owner` | 软件包的所有者 | | `owner` | 软件包的所有者 |
| `group` | 软件包自定义分组名称 |
使用HTTP基本身份验证的示例请求 使用HTTP基本身份验证的示例请求
```shell ```shell
curl --user your_username:your_password_or_token \ curl --user your_username:your_password_or_token \
--upload-file path/to/file.rpm \ --upload-file path/to/file.rpm \
https://gitea.example.com/api/packages/testuser/rpm/upload https://gitea.example.com/api/packages/testuser/rpm/centos/el7/version/upload
``` ```
如果您使用 2FA 或 OAuth请使用[个人访问令牌](development/api-usage.md#通过-api-认证)替代密码。您无法将具有相同名称的文件两次发布到软件包中。您必须先删除现有的软件包版本。 如果您使用 2FA 或 OAuth请使用[个人访问令牌](development/api-usage.md#通过-api-认证)替代密码。您无法将具有相同名称的文件两次发布到软件包中。您必须先删除现有的软件包版本。
@ -77,12 +79,13 @@ curl --user your_username:your_password_or_token \
要删除 RPM 软件包,请执行 HTTP `DELETE` 操作。如果没有文件剩余,这也将删除软件包版本。 要删除 RPM 软件包,请执行 HTTP `DELETE` 操作。如果没有文件剩余,这也将删除软件包版本。
``` ```
DELETE https://gitea.example.com/api/packages/{owner}/rpm/{package_name}/{package_version}/{architecture} DELETE https://gitea.example.com/api/packages/{owner}/rpm/{group}/package/{package_name}/{package_version}/{architecture}
``` ```
| 参数 | 描述 | | 参数 | 描述 |
| ----------------- | -------------- | | ----------------- | -------------- |
| `owner` | 软件包的所有者 | | `owner` | 软件包的所有者 |
| `group` | 软件包自定义分组 |
| `package_name` | 软件包名称 | | `package_name` | 软件包名称 |
| `package_version` | 软件包版本 | | `package_version` | 软件包版本 |
| `architecture` | 软件包架构 | | `architecture` | 软件包架构 |
@ -91,7 +94,7 @@ DELETE https://gitea.example.com/api/packages/{owner}/rpm/{package_name}/{packag
```shell ```shell
curl --user your_username:your_token_or_password -X DELETE \ curl --user your_username:your_token_or_password -X DELETE \
https://gitea.example.com/api/packages/testuser/rpm/test-package/1.0.0/x86_64 https://gitea.example.com/api/packages/testuser/rpm/centos/el7/package/test-package/1.0.0/x86_64
``` ```
服务器将以以下HTTP状态码响应 服务器将以以下HTTP状态码响应

View file

@ -15,8 +15,7 @@ import (
) )
const ( const (
PropertyMetadata = "rpm.metadata" PropertyMetadata = "rpm.metadata"
SettingKeyPrivate = "rpm.key.private" SettingKeyPrivate = "rpm.key.private"
SettingKeyPublic = "rpm.key.public" SettingKeyPublic = "rpm.key.public"

View file

@ -4,6 +4,7 @@
package templates package templates
import ( import (
"regexp"
"strings" "strings"
"code.gitea.io/gitea/modules/base" "code.gitea.io/gitea/modules/base"
@ -25,6 +26,10 @@ func (su *StringUtils) Contains(s, substr string) bool {
return strings.Contains(s, substr) return strings.Contains(s, substr)
} }
func (su *StringUtils) ReplaceAllStringRegex(s, regex, new string) string {
return regexp.MustCompile(regex).ReplaceAllString(s, new)
}
func (su *StringUtils) Split(s, sep string) []string { func (su *StringUtils) Split(s, sep string) []string {
return strings.Split(s, sep) return strings.Split(s, sep)
} }

View file

@ -512,19 +512,7 @@ func CommonRoutes() *web.Route {
r.Get("/files/{id}/{version}/{filename}", pypi.DownloadPackageFile) r.Get("/files/{id}/{version}/{filename}", pypi.DownloadPackageFile)
r.Get("/simple/{id}", pypi.PackageMetadata) r.Get("/simple/{id}", pypi.PackageMetadata)
}, reqPackageAccess(perm.AccessModeRead)) }, reqPackageAccess(perm.AccessModeRead))
r.Group("/rpm", func() { r.Group("/rpm", RpmRoutes(r), reqPackageAccess(perm.AccessModeRead))
r.Get(".repo", rpm.GetRepositoryConfig)
r.Get("/repository.key", rpm.GetRepositoryKey)
r.Put("/upload", reqPackageAccess(perm.AccessModeWrite), rpm.UploadPackageFile)
r.Group("/package/{name}/{version}/{architecture}", func() {
r.Get("", rpm.DownloadPackageFile)
r.Delete("", reqPackageAccess(perm.AccessModeWrite), rpm.DeletePackageFile)
})
r.Group("/repodata/{filename}", func() {
r.Head("", rpm.CheckRepositoryFileExistence)
r.Get("", rpm.GetRepositoryFile)
})
}, reqPackageAccess(perm.AccessModeRead))
r.Group("/rubygems", func() { r.Group("/rubygems", func() {
r.Get("/specs.4.8.gz", rubygems.EnumeratePackages) r.Get("/specs.4.8.gz", rubygems.EnumeratePackages)
r.Get("/latest_specs.4.8.gz", rubygems.EnumeratePackagesLatest) r.Get("/latest_specs.4.8.gz", rubygems.EnumeratePackagesLatest)
@ -589,6 +577,82 @@ func CommonRoutes() *web.Route {
return r return r
} }
// Support for uploading rpm packages with arbitrary depth paths
func RpmRoutes(r *web.Route) func() {
var (
groupRepoInfo = regexp.MustCompile(`\A((?:/(?:[^/]+))*|)\.repo\z`)
groupUpload = regexp.MustCompile(`\A((?:/(?:[^/]+))*|)/upload\z`)
groupRpm = regexp.MustCompile(`\A((?:/(?:[^/]+))*|)/package/([^/]+)/([^/]+)/([^/]+)(?:/([^/]+\.rpm)|)\z`)
groupMetadata = regexp.MustCompile(`\A((?:/(?:[^/]+))*|)/repodata/([^/]+)\z`)
)
return func() {
r.Methods("HEAD,GET,POST,PUT,PATCH,DELETE", "*", func(ctx *context.Context) {
path := ctx.Params("*")
isHead := ctx.Req.Method == "HEAD"
isGetHead := ctx.Req.Method == "HEAD" || ctx.Req.Method == "GET"
isPut := ctx.Req.Method == "PUT"
isDelete := ctx.Req.Method == "DELETE"
if path == "/repository.key" && isGetHead {
rpm.GetRepositoryKey(ctx)
return
}
// get repo
m := groupRepoInfo.FindStringSubmatch(path)
if len(m) == 2 && isGetHead {
ctx.SetParams("group", strings.Trim(m[1], "/"))
rpm.GetRepositoryConfig(ctx)
return
}
// get meta
m = groupMetadata.FindStringSubmatch(path)
if len(m) == 3 && isGetHead {
ctx.SetParams("group", strings.Trim(m[1], "/"))
ctx.SetParams("filename", m[2])
if isHead {
rpm.CheckRepositoryFileExistence(ctx)
} else {
rpm.GetRepositoryFile(ctx)
}
return
}
// upload
m = groupUpload.FindStringSubmatch(path)
if len(m) == 2 && isPut {
reqPackageAccess(perm.AccessModeWrite)(ctx)
if ctx.Written() {
return
}
ctx.SetParams("group", strings.Trim(m[1], "/"))
rpm.UploadPackageFile(ctx)
return
}
// rpm down/delete
m = groupRpm.FindStringSubmatch(path)
if len(m) == 6 {
ctx.SetParams("group", strings.Trim(m[1], "/"))
ctx.SetParams("name", m[2])
ctx.SetParams("version", m[3])
ctx.SetParams("architecture", m[4])
if isGetHead {
rpm.DownloadPackageFile(ctx)
return
} else if isDelete {
reqPackageAccess(perm.AccessModeWrite)(ctx)
if ctx.Written() {
return
}
rpm.DeletePackageFile(ctx)
}
}
// default
ctx.Status(http.StatusNotFound)
})
}
}
// ContainerRoutes provides endpoints that implement the OCI API to serve containers // ContainerRoutes provides endpoints that implement the OCI API to serve containers
// These have to be mounted on `/v2/...` to comply with the OCI spec: // These have to be mounted on `/v2/...` to comply with the OCI spec:
// https://github.com/opencontainers/distribution-spec/blob/main/spec.md // https://github.com/opencontainers/distribution-spec/blob/main/spec.md

View file

@ -33,11 +33,14 @@ func apiError(ctx *context.Context, status int, obj any) {
// https://dnf.readthedocs.io/en/latest/conf_ref.html // https://dnf.readthedocs.io/en/latest/conf_ref.html
func GetRepositoryConfig(ctx *context.Context) { func GetRepositoryConfig(ctx *context.Context) {
group := ctx.Params("group")
if group != "" {
group = fmt.Sprintf("/%s", group)
}
url := fmt.Sprintf("%sapi/packages/%s/rpm", setting.AppURL, ctx.Package.Owner.Name) url := fmt.Sprintf("%sapi/packages/%s/rpm", setting.AppURL, ctx.Package.Owner.Name)
ctx.PlainText(http.StatusOK, `[gitea-`+ctx.Package.Owner.LowerName+strings.ReplaceAll(group, "/", "-")+`]
ctx.PlainText(http.StatusOK, `[gitea-`+ctx.Package.Owner.LowerName+`] name=`+ctx.Package.Owner.Name+` - `+setting.AppName+strings.ReplaceAll(group, "/", " - ")+`
name=`+ctx.Package.Owner.Name+` - `+setting.AppName+` baseurl=`+url+group+`/
baseurl=`+url+`
enabled=1 enabled=1
gpgcheck=1 gpgcheck=1
gpgkey=`+url+`/repository.key`) gpgkey=`+url+`/repository.key`)
@ -64,7 +67,7 @@ func CheckRepositoryFileExistence(ctx *context.Context) {
return return
} }
pf, err := packages_model.GetFileForVersionByName(ctx, pv.ID, ctx.Params("filename"), packages_model.EmptyFileKey) pf, err := packages_model.GetFileForVersionByName(ctx, pv.ID, ctx.Params("filename"), ctx.Params("group"))
if err != nil { if err != nil {
if errors.Is(err, util.ErrNotExist) { if errors.Is(err, util.ErrNotExist) {
ctx.Status(http.StatusNotFound) ctx.Status(http.StatusNotFound)
@ -93,7 +96,8 @@ func GetRepositoryFile(ctx *context.Context) {
ctx, ctx,
pv, pv,
&packages_service.PackageFileInfo{ &packages_service.PackageFileInfo{
Filename: ctx.Params("filename"), Filename: ctx.Params("filename"),
CompositeKey: ctx.Params("group"),
}, },
) )
if err != nil { if err != nil {
@ -145,7 +149,7 @@ func UploadPackageFile(ctx *context.Context) {
apiError(ctx, http.StatusInternalServerError, err) apiError(ctx, http.StatusInternalServerError, err)
return return
} }
group := ctx.Params("group")
_, _, err = packages_service.CreatePackageOrAddFileToExisting( _, _, err = packages_service.CreatePackageOrAddFileToExisting(
ctx, ctx,
&packages_service.PackageCreationInfo{ &packages_service.PackageCreationInfo{
@ -153,14 +157,15 @@ func UploadPackageFile(ctx *context.Context) {
Owner: ctx.Package.Owner, Owner: ctx.Package.Owner,
PackageType: packages_model.TypeRpm, PackageType: packages_model.TypeRpm,
Name: pck.Name, Name: pck.Name,
Version: pck.Version, Version: strings.Trim(fmt.Sprintf("%s/%s", group, pck.Version), "/"),
}, },
Creator: ctx.Doer, Creator: ctx.Doer,
Metadata: pck.VersionMetadata, Metadata: pck.VersionMetadata,
}, },
&packages_service.PackageFileCreationInfo{ &packages_service.PackageFileCreationInfo{
PackageFileInfo: packages_service.PackageFileInfo{ PackageFileInfo: packages_service.PackageFileInfo{
Filename: fmt.Sprintf("%s-%s.%s.rpm", pck.Name, pck.Version, pck.FileMetadata.Architecture), Filename: fmt.Sprintf("%s-%s.%s.rpm", pck.Name, pck.Version, pck.FileMetadata.Architecture),
CompositeKey: group,
}, },
Creator: ctx.Doer, Creator: ctx.Doer,
Data: buf, Data: buf,
@ -182,7 +187,7 @@ func UploadPackageFile(ctx *context.Context) {
return return
} }
if err := rpm_service.BuildRepositoryFiles(ctx, ctx.Package.Owner.ID); err != nil { if err := rpm_service.BuildRepositoryFiles(ctx, ctx.Package.Owner.ID, group); err != nil {
apiError(ctx, http.StatusInternalServerError, err) apiError(ctx, http.StatusInternalServerError, err)
return return
} }
@ -191,19 +196,20 @@ func UploadPackageFile(ctx *context.Context) {
} }
func DownloadPackageFile(ctx *context.Context) { func DownloadPackageFile(ctx *context.Context) {
group := ctx.Params("group")
name := ctx.Params("name") name := ctx.Params("name")
version := ctx.Params("version") version := ctx.Params("version")
s, u, pf, err := packages_service.GetFileStreamByPackageNameAndVersion( s, u, pf, err := packages_service.GetFileStreamByPackageNameAndVersion(
ctx, ctx,
&packages_service.PackageInfo{ &packages_service.PackageInfo{
Owner: ctx.Package.Owner, Owner: ctx.Package.Owner,
PackageType: packages_model.TypeRpm, PackageType: packages_model.TypeRpm,
Name: name, Name: name,
Version: version, Version: strings.Trim(fmt.Sprintf("%s/%s", group, version), "/"),
}, },
&packages_service.PackageFileInfo{ &packages_service.PackageFileInfo{
Filename: fmt.Sprintf("%s-%s.%s.rpm", name, version, ctx.Params("architecture")), Filename: fmt.Sprintf("%s-%s.%s.rpm", name, version, ctx.Params("architecture")),
CompositeKey: group,
}, },
) )
if err != nil { if err != nil {
@ -219,14 +225,19 @@ func DownloadPackageFile(ctx *context.Context) {
} }
func DeletePackageFile(webctx *context.Context) { func DeletePackageFile(webctx *context.Context) {
group := webctx.Params("group")
name := webctx.Params("name") name := webctx.Params("name")
version := webctx.Params("version") version := webctx.Params("version")
architecture := webctx.Params("architecture") architecture := webctx.Params("architecture")
var pd *packages_model.PackageDescriptor var pd *packages_model.PackageDescriptor
err := db.WithTx(webctx, func(ctx stdctx.Context) error { err := db.WithTx(webctx, func(ctx stdctx.Context) error {
pv, err := packages_model.GetVersionByNameAndVersion(ctx, webctx.Package.Owner.ID, packages_model.TypeRpm, name, version) pv, err := packages_model.GetVersionByNameAndVersion(ctx,
webctx.Package.Owner.ID,
packages_model.TypeRpm,
name,
strings.Trim(fmt.Sprintf("%s/%s", group, version), "/"),
)
if err != nil { if err != nil {
return err return err
} }
@ -235,7 +246,7 @@ func DeletePackageFile(webctx *context.Context) {
ctx, ctx,
pv.ID, pv.ID,
fmt.Sprintf("%s-%s.%s.rpm", name, version, architecture), fmt.Sprintf("%s-%s.%s.rpm", name, version, architecture),
packages_model.EmptyFileKey, group,
) )
if err != nil { if err != nil {
return err return err
@ -275,7 +286,7 @@ func DeletePackageFile(webctx *context.Context) {
notify_service.PackageDelete(webctx, webctx.Doer, pd) notify_service.PackageDelete(webctx, webctx.Doer, pd)
} }
if err := rpm_service.BuildRepositoryFiles(webctx, webctx.Package.Owner.ID); err != nil { if err := rpm_service.BuildRepositoryFiles(webctx, webctx.Package.Owner.ID, group); err != nil {
apiError(webctx, http.StatusInternalServerError, err) apiError(webctx, http.StatusInternalServerError, err)
return return
} }

View file

@ -125,17 +125,18 @@ type packageData struct {
type packageCache = map[*packages_model.PackageFile]*packageData type packageCache = map[*packages_model.PackageFile]*packageData
// BuildRepositoryFiles builds metadata files for the repository // BuildSpecificRepositoryFiles builds metadata files for the repository
func BuildRepositoryFiles(ctx context.Context, ownerID int64) error { func BuildRepositoryFiles(ctx context.Context, ownerID int64, compositeKey string) error {
pv, err := GetOrCreateRepositoryVersion(ctx, ownerID) pv, err := GetOrCreateRepositoryVersion(ctx, ownerID)
if err != nil { if err != nil {
return err return err
} }
pfs, _, err := packages_model.SearchFiles(ctx, &packages_model.PackageFileSearchOptions{ pfs, _, err := packages_model.SearchFiles(ctx, &packages_model.PackageFileSearchOptions{
OwnerID: ownerID, OwnerID: ownerID,
PackageType: packages_model.TypeRpm, PackageType: packages_model.TypeRpm,
Query: "%.rpm", Query: "%.rpm",
CompositeKey: compositeKey,
}) })
if err != nil { if err != nil {
return err return err
@ -194,15 +195,15 @@ func BuildRepositoryFiles(ctx context.Context, ownerID int64) error {
cache[pf] = pd cache[pf] = pd
} }
primary, err := buildPrimary(ctx, pv, pfs, cache) primary, err := buildPrimary(ctx, pv, pfs, cache, compositeKey)
if err != nil { if err != nil {
return err return err
} }
filelists, err := buildFilelists(ctx, pv, pfs, cache) filelists, err := buildFilelists(ctx, pv, pfs, cache, compositeKey)
if err != nil { if err != nil {
return err return err
} }
other, err := buildOther(ctx, pv, pfs, cache) other, err := buildOther(ctx, pv, pfs, cache, compositeKey)
if err != nil { if err != nil {
return err return err
} }
@ -216,11 +217,12 @@ func BuildRepositoryFiles(ctx context.Context, ownerID int64) error {
filelists, filelists,
other, other,
}, },
compositeKey,
) )
} }
// https://docs.pulpproject.org/en/2.19/plugins/pulp_rpm/tech-reference/rpm.html#repomd-xml // https://docs.pulpproject.org/en/2.19/plugins/pulp_rpm/tech-reference/rpm.html#repomd-xml
func buildRepomd(ctx context.Context, pv *packages_model.PackageVersion, ownerID int64, data []*repoData) error { func buildRepomd(ctx context.Context, pv *packages_model.PackageVersion, ownerID int64, data []*repoData, compositeKey string) error {
type Repomd struct { type Repomd struct {
XMLName xml.Name `xml:"repomd"` XMLName xml.Name `xml:"repomd"`
Xmlns string `xml:"xmlns,attr"` Xmlns string `xml:"xmlns,attr"`
@ -275,7 +277,8 @@ func buildRepomd(ctx context.Context, pv *packages_model.PackageVersion, ownerID
pv, pv,
&packages_service.PackageFileCreationInfo{ &packages_service.PackageFileCreationInfo{
PackageFileInfo: packages_service.PackageFileInfo{ PackageFileInfo: packages_service.PackageFileInfo{
Filename: file.Name, Filename: file.Name,
CompositeKey: compositeKey,
}, },
Creator: user_model.NewGhostUser(), Creator: user_model.NewGhostUser(),
Data: file.Data, Data: file.Data,
@ -292,7 +295,7 @@ func buildRepomd(ctx context.Context, pv *packages_model.PackageVersion, ownerID
} }
// https://docs.pulpproject.org/en/2.19/plugins/pulp_rpm/tech-reference/rpm.html#primary-xml // https://docs.pulpproject.org/en/2.19/plugins/pulp_rpm/tech-reference/rpm.html#primary-xml
func buildPrimary(ctx context.Context, pv *packages_model.PackageVersion, pfs []*packages_model.PackageFile, c packageCache) (*repoData, error) { func buildPrimary(ctx context.Context, pv *packages_model.PackageVersion, pfs []*packages_model.PackageFile, c packageCache, compositeKey string) (*repoData, error) {
type Version struct { type Version struct {
Epoch string `xml:"epoch,attr"` Epoch string `xml:"epoch,attr"`
Version string `xml:"ver,attr"` Version string `xml:"ver,attr"`
@ -372,7 +375,7 @@ func buildPrimary(ctx context.Context, pv *packages_model.PackageVersion, pfs []
files = append(files, f) files = append(files, f)
} }
} }
packageVersion := fmt.Sprintf("%s-%s", pd.FileMetadata.Version, pd.FileMetadata.Release)
packages = append(packages, &Package{ packages = append(packages, &Package{
Type: "rpm", Type: "rpm",
Name: pd.Package.Name, Name: pd.Package.Name,
@ -401,7 +404,7 @@ func buildPrimary(ctx context.Context, pv *packages_model.PackageVersion, pfs []
Archive: pd.FileMetadata.ArchiveSize, Archive: pd.FileMetadata.ArchiveSize,
}, },
Location: Location{ Location: Location{
Href: fmt.Sprintf("package/%s/%s/%s", url.PathEscape(pd.Package.Name), url.PathEscape(pd.Version.Version), url.PathEscape(pd.FileMetadata.Architecture)), Href: fmt.Sprintf("package/%s/%s/%s/%s", url.PathEscape(pd.Package.Name), url.PathEscape(packageVersion), url.PathEscape(pd.FileMetadata.Architecture), url.PathEscape(fmt.Sprintf("%s-%s.%s.rpm", pd.Package.Name, packageVersion, pd.FileMetadata.Architecture))),
}, },
Format: Format{ Format: Format{
License: pd.VersionMetadata.License, License: pd.VersionMetadata.License,
@ -431,11 +434,11 @@ func buildPrimary(ctx context.Context, pv *packages_model.PackageVersion, pfs []
XmlnsRpm: "http://linux.duke.edu/metadata/rpm", XmlnsRpm: "http://linux.duke.edu/metadata/rpm",
PackageCount: len(pfs), PackageCount: len(pfs),
Packages: packages, Packages: packages,
}) }, compositeKey)
} }
// https://docs.pulpproject.org/en/2.19/plugins/pulp_rpm/tech-reference/rpm.html#filelists-xml // https://docs.pulpproject.org/en/2.19/plugins/pulp_rpm/tech-reference/rpm.html#filelists-xml
func buildFilelists(ctx context.Context, pv *packages_model.PackageVersion, pfs []*packages_model.PackageFile, c packageCache) (*repoData, error) { //nolint:dupl func buildFilelists(ctx context.Context, pv *packages_model.PackageVersion, pfs []*packages_model.PackageFile, c packageCache, compositeKey string) (*repoData, error) { //nolint:dupl
type Version struct { type Version struct {
Epoch string `xml:"epoch,attr"` Epoch string `xml:"epoch,attr"`
Version string `xml:"ver,attr"` Version string `xml:"ver,attr"`
@ -478,11 +481,12 @@ func buildFilelists(ctx context.Context, pv *packages_model.PackageVersion, pfs
Xmlns: "http://linux.duke.edu/metadata/other", Xmlns: "http://linux.duke.edu/metadata/other",
PackageCount: len(pfs), PackageCount: len(pfs),
Packages: packages, Packages: packages,
}) },
compositeKey)
} }
// https://docs.pulpproject.org/en/2.19/plugins/pulp_rpm/tech-reference/rpm.html#other-xml // https://docs.pulpproject.org/en/2.19/plugins/pulp_rpm/tech-reference/rpm.html#other-xml
func buildOther(ctx context.Context, pv *packages_model.PackageVersion, pfs []*packages_model.PackageFile, c packageCache) (*repoData, error) { //nolint:dupl func buildOther(ctx context.Context, pv *packages_model.PackageVersion, pfs []*packages_model.PackageFile, c packageCache, compositeKey string) (*repoData, error) { //nolint:dupl
type Version struct { type Version struct {
Epoch string `xml:"epoch,attr"` Epoch string `xml:"epoch,attr"`
Version string `xml:"ver,attr"` Version string `xml:"ver,attr"`
@ -525,7 +529,7 @@ func buildOther(ctx context.Context, pv *packages_model.PackageVersion, pfs []*p
Xmlns: "http://linux.duke.edu/metadata/other", Xmlns: "http://linux.duke.edu/metadata/other",
PackageCount: len(pfs), PackageCount: len(pfs),
Packages: packages, Packages: packages,
}) }, compositeKey)
} }
// writtenCounter counts all written bytes // writtenCounter counts all written bytes
@ -545,10 +549,8 @@ func (wc *writtenCounter) Written() int64 {
return wc.written return wc.written
} }
func addDataAsFileToRepo(ctx context.Context, pv *packages_model.PackageVersion, filetype string, obj any) (*repoData, error) { func addDataAsFileToRepo(ctx context.Context, pv *packages_model.PackageVersion, filetype string, obj any, compositeKey string) (*repoData, error) {
content, _ := packages_module.NewHashedBuffer() content, _ := packages_module.NewHashedBuffer()
defer content.Close()
gzw := gzip.NewWriter(content) gzw := gzip.NewWriter(content)
wc := &writtenCounter{} wc := &writtenCounter{}
h := sha256.New() h := sha256.New()
@ -571,7 +573,8 @@ func addDataAsFileToRepo(ctx context.Context, pv *packages_model.PackageVersion,
pv, pv,
&packages_service.PackageFileCreationInfo{ &packages_service.PackageFileCreationInfo{
PackageFileInfo: packages_service.PackageFileInfo{ PackageFileInfo: packages_service.PackageFileInfo{
Filename: filename, Filename: filename,
CompositeKey: compositeKey,
}, },
Creator: user_model.NewGhostUser(), Creator: user_model.NewGhostUser(),
Data: content, Data: content,

View file

@ -4,19 +4,23 @@
<div class="ui form"> <div class="ui form">
<div class="field"> <div class="field">
<label>{{svg "octicon-terminal"}} {{ctx.Locale.Tr "packages.rpm.registry"}}</label> <label>{{svg "octicon-terminal"}} {{ctx.Locale.Tr "packages.rpm.registry"}}</label>
<div class="markup"><pre class="code-block"><code># {{ctx.Locale.Tr "packages.rpm.distro.redhat"}} <div class="markup"><pre class="code-block"><code># {{ctx.Locale.Tr "packages.rpm.distros.redhat"}}
dnf config-manager --add-repo <gitea-origin-url data-url="{{AppSubUrl}}/api/packages/{{$.PackageDescriptor.Owner.Name}}/rpm.repo"></gitea-origin-url> {{$group_name:= StringUtils.ReplaceAllStringRegex .PackageDescriptor.Version.Version "(/[^/]+|[^/]*)\\z" "" -}}
{{- if $group_name -}}
{{- $group_name = (print "/" $group_name) -}}
{{- end -}}
dnf config-manager --add-repo <gitea-origin-url data-url="{{AppSubUrl}}/api/packages/{{$.PackageDescriptor.Owner.Name}}/rpm{{$group_name}}.repo"></gitea-origin-url>
# {{ctx.Locale.Tr "packages.rpm.distro.suse"}} # {{ctx.Locale.Tr "packages.rpm.distros.suse"}}
zypper addrepo <gitea-origin-url data-url="{{AppSubUrl}}/api/packages/{{$.PackageDescriptor.Owner.Name}}/rpm.repo"></gitea-origin-url></code></pre></div> zypper addrepo <gitea-origin-url data-url="{{AppSubUrl}}/api/packages/{{$.PackageDescriptor.Owner.Name}}/rpm{{$group_name}}.repo"></gitea-origin-url></code></pre></div>
</div> </div>
<div class="field"> <div class="field">
<label>{{svg "octicon-terminal"}} {{ctx.Locale.Tr "packages.rpm.install"}}</label> <label>{{svg "octicon-terminal"}} {{ctx.Locale.Tr "packages.rpm.install"}}</label>
<div class="markup"> <div class="markup">
<pre class="code-block"><code># {{ctx.Locale.Tr "packages.rpm.distro.redhat"}} <pre class="code-block"><code># {{ctx.Locale.Tr "packages.rpm.distros.redhat"}}
dnf install {{$.PackageDescriptor.Package.Name}} dnf install {{$.PackageDescriptor.Package.Name}}
# {{ctx.Locale.Tr "packages.rpm.distro.suse"}} # {{ctx.Locale.Tr "packages.rpm.distros.suse"}}
zypper install {{$.PackageDescriptor.Package.Name}}</code></pre> zypper install {{$.PackageDescriptor.Package.Name}}</code></pre>
</div> </div>
</div> </div>

View file

@ -76,12 +76,12 @@ Mu0UFYgZ/bYnuvn/vz4wtCz8qMwsHUvP0PX3tbYFUctAPdrY6tiiDtcCddDECahx7SuVNP5dpmb5
t.Run("RepositoryConfig", func(t *testing.T) { t.Run("RepositoryConfig", func(t *testing.T) {
defer tests.PrintCurrentTest(t)() defer tests.PrintCurrentTest(t)()
req := NewRequest(t, "GET", rootURL+".repo") req := NewRequest(t, "GET", rootURL+"/el9/stable.repo")
resp := MakeRequest(t, req, http.StatusOK) resp := MakeRequest(t, req, http.StatusOK)
expected := fmt.Sprintf(`[gitea-%s] expected := fmt.Sprintf(`[gitea-%s-el9-stable]
name=%s - %s name=%s - %s - el9 - stable
baseurl=%sapi/packages/%s/rpm baseurl=%sapi/packages/%s/rpm/el9/stable/
enabled=1 enabled=1
gpgcheck=1 gpgcheck=1
gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppName, setting.AppURL, user.Name, setting.AppURL, user.Name) gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppName, setting.AppURL, user.Name, setting.AppURL, user.Name)
@ -100,7 +100,7 @@ gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppN
}) })
t.Run("Upload", func(t *testing.T) { t.Run("Upload", func(t *testing.T) {
url := rootURL + "/upload" url := rootURL + "/el9/stable/upload"
req := NewRequestWithBody(t, "PUT", url, bytes.NewReader(content)) req := NewRequestWithBody(t, "PUT", url, bytes.NewReader(content))
MakeRequest(t, req, http.StatusUnauthorized) MakeRequest(t, req, http.StatusUnauthorized)
@ -118,7 +118,7 @@ gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppN
assert.Nil(t, pd.SemVer) assert.Nil(t, pd.SemVer)
assert.IsType(t, &rpm_module.VersionMetadata{}, pd.Metadata) assert.IsType(t, &rpm_module.VersionMetadata{}, pd.Metadata)
assert.Equal(t, packageName, pd.Package.Name) assert.Equal(t, packageName, pd.Package.Name)
assert.Equal(t, packageVersion, pd.Version.Version) assert.Equal(t, fmt.Sprintf("el9/stable/%s", packageVersion), pd.Version.Version)
pfs, err := packages.GetFilesByVersionID(db.DefaultContext, pvs[0].ID) pfs, err := packages.GetFilesByVersionID(db.DefaultContext, pvs[0].ID)
assert.NoError(t, err) assert.NoError(t, err)
@ -138,7 +138,7 @@ gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppN
t.Run("Download", func(t *testing.T) { t.Run("Download", func(t *testing.T) {
defer tests.PrintCurrentTest(t)() defer tests.PrintCurrentTest(t)()
req := NewRequest(t, "GET", fmt.Sprintf("%s/package/%s/%s/%s", rootURL, packageName, packageVersion, packageArchitecture)) req := NewRequest(t, "GET", fmt.Sprintf("%s/el9/stable/package/%s/%s/%s", rootURL, packageName, packageVersion, packageArchitecture))
resp := MakeRequest(t, req, http.StatusOK) resp := MakeRequest(t, req, http.StatusOK)
assert.Equal(t, content, resp.Body.Bytes()) assert.Equal(t, content, resp.Body.Bytes())
@ -147,7 +147,7 @@ gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppN
t.Run("Repository", func(t *testing.T) { t.Run("Repository", func(t *testing.T) {
defer tests.PrintCurrentTest(t)() defer tests.PrintCurrentTest(t)()
url := rootURL + "/repodata" url := rootURL + "/el9/stable/repodata"
req := NewRequest(t, "HEAD", url+"/dummy.xml") req := NewRequest(t, "HEAD", url+"/dummy.xml")
MakeRequest(t, req, http.StatusNotFound) MakeRequest(t, req, http.StatusNotFound)
@ -201,8 +201,8 @@ gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppN
switch d.Type { switch d.Type {
case "primary": case "primary":
assert.EqualValues(t, 718, d.Size) assert.EqualValues(t, 722, d.Size)
assert.EqualValues(t, 1729, d.OpenSize) assert.EqualValues(t, 1759, d.OpenSize)
assert.Equal(t, "repodata/primary.xml.gz", d.Location.Href) assert.Equal(t, "repodata/primary.xml.gz", d.Location.Href)
case "filelists": case "filelists":
assert.EqualValues(t, 257, d.Size) assert.EqualValues(t, 257, d.Size)
@ -311,7 +311,7 @@ gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppN
assert.EqualValues(t, len(content), p.Size.Package) assert.EqualValues(t, len(content), p.Size.Package)
assert.EqualValues(t, 13, p.Size.Installed) assert.EqualValues(t, 13, p.Size.Installed)
assert.EqualValues(t, 272, p.Size.Archive) assert.EqualValues(t, 272, p.Size.Archive)
assert.Equal(t, fmt.Sprintf("package/%s/%s/%s", packageName, packageVersion, packageArchitecture), p.Location.Href) assert.Equal(t, fmt.Sprintf("package/%s/%s/%s/%s", packageName, packageVersion, packageArchitecture, fmt.Sprintf("%s-%s.%s.rpm", packageName, packageVersion, packageArchitecture)), p.Location.Href)
f := p.Format f := p.Format
assert.Equal(t, "MIT", f.License) assert.Equal(t, "MIT", f.License)
assert.Len(t, f.Provides.Entries, 2) assert.Len(t, f.Provides.Entries, 2)
@ -401,18 +401,17 @@ gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppN
t.Run("Delete", func(t *testing.T) { t.Run("Delete", func(t *testing.T) {
defer tests.PrintCurrentTest(t)() defer tests.PrintCurrentTest(t)()
req := NewRequest(t, "DELETE", fmt.Sprintf("%s/package/%s/%s/%s", rootURL, packageName, packageVersion, packageArchitecture)) req := NewRequest(t, "DELETE", fmt.Sprintf("%s/el9/stable/package/%s/%s/%s", rootURL, packageName, packageVersion, packageArchitecture))
MakeRequest(t, req, http.StatusUnauthorized) MakeRequest(t, req, http.StatusUnauthorized)
req = NewRequest(t, "DELETE", fmt.Sprintf("%s/package/%s/%s/%s", rootURL, packageName, packageVersion, packageArchitecture)). req = NewRequest(t, "DELETE", fmt.Sprintf("%s/el9/stable/package/%s/%s/%s", rootURL, packageName, packageVersion, packageArchitecture)).
AddBasicAuth(user.Name) AddBasicAuth(user.Name)
MakeRequest(t, req, http.StatusNoContent) MakeRequest(t, req, http.StatusNoContent)
pvs, err := packages.GetVersionsByPackageType(db.DefaultContext, user.ID, packages.TypeRpm) pvs, err := packages.GetVersionsByPackageType(db.DefaultContext, user.ID, packages.TypeRpm)
assert.NoError(t, err) assert.NoError(t, err)
assert.Empty(t, pvs) assert.Empty(t, pvs)
req = NewRequest(t, "DELETE", fmt.Sprintf("%s/el9/stable/package/%s/%s/%s", rootURL, packageName, packageVersion, packageArchitecture)).
req = NewRequest(t, "DELETE", fmt.Sprintf("%s/package/%s/%s/%s", rootURL, packageName, packageVersion, packageArchitecture)).
AddBasicAuth(user.Name) AddBasicAuth(user.Name)
MakeRequest(t, req, http.StatusNotFound) MakeRequest(t, req, http.StatusNotFound)
}) })