Create, Sync and Publish a Repository

One of the most common workflows is a fetching content from a remote source and making it available for users.

Create an RPM repository foo

#!/usr/bin/env bash

# Create RPM repository
if [ $# -eq 0 ]; then
  export REPO_NAME="foo"
else
  export REPO_NAME="$1"
fi

echo "Creating a new repository named $REPO_NAME."
export REPO_HREF=$(http POST $BASE_ADDR/pulp/api/v3/repositories/rpm/rpm/ name=$REPO_NAME \
    | jq -r '.pulp_href')

echo "Inspecting Repository."
http $BASE_ADDR$REPO_HREF

Repository GET response:

{
    "description": null,
    "latest_version_href": "/pulp/api/v3/repositories/rpm/rpm/a02ace53-d490-458d-8b93-604fbcd23a9c/versions/0/",
    "name": "foo",
    "pulp_created": "2019-11-27T13:30:28.159167Z",
    "pulp_href": "/pulp/api/v3/repositories/rpm/rpm/a02ace53-d490-458d-8b93-604fbcd23a9c/",
    "versions_href": "/pulp/api/v3/repositories/rpm/rpm/a02ace53-d490-458d-8b93-604fbcd23a9c/versions/"
}

Create a new remote bar

By default policy='immediate which means that all the content is downloaded right away. Specify policy='on_demand' to make synchronization of a repository faster and only to download RPMs whenever they are requested by clients.

#!/usr/bin/env bash

# Create new RPM remote
echo "Creating a remote that points to an external source of files."
http POST $BASE_ADDR/pulp/api/v3/remotes/rpm/rpm/ \
    name='bar' \
    url='https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-unsigned/' \
    policy='on_demand'

echo "Export an environment variable for the new remote URI."
export REMOTE_HREF=$(http $BASE_ADDR/pulp/api/v3/remotes/rpm/rpm/ \
    | jq -r '.results[] | select(.name == "bar") | .pulp_href')

echo "Inspecting new Remote."
http $BASE_ADDR$REMOTE_HREF

Remote GET response:

{
    "ca_cert": null,
    "client_cert": null,
    "client_key": null,
    "download_concurrency": 20,
    "name": "bar",
    "policy": "on_demand",
    "proxy_url": null,
    "pulp_created": "2019-11-27T13:30:29.199173Z",
    "pulp_href": "/pulp/api/v3/remotes/rpm/rpm/2ceb5262-a5b2-4297-afdf-a31f7e46dfc5/",
    "pulp_last_updated": "2019-11-27T13:30:29.199187Z",
    "tls_validation": true,
    "url": "https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm/"
}

Sync repository foo using remote bar

#!/usr/bin/env bash

# Sync repository foo using remote bar
echo "Create a task to sync the repository using the remote."
export TASK_URL=$(http POST $BASE_ADDR$REPO_HREF'sync/' \
    remote=$REMOTE_HREF \
    | jq -r '.task')

# Poll the task (here we use a function defined in docs/_scripts/base.sh)
wait_until_task_finished $BASE_ADDR$TASK_URL

# After the task is complete, it gives us a new repository version
echo "Set REPOVERSION_HREF from finished task."
export REPOVERSION_HREF=$(http $BASE_ADDR$TASK_URL| jq -r '.created_resources | first')

echo "Inspecting RepositoryVersion."
http $BASE_ADDR$REPOVERSION_HREF

You can specify mirror=True for a mirror mode. It means Pulp won’t update repository using previous repository version but create a new copy of remote repository as a new repository version.

Optionally, you can skip SRPM packages by using skip_types:="[\"srpm\"]" option.

By default, optimize=True and sync will only proceed if changes are present. You can override this by setting optimize=False which will disable optimizations and run a full sync.

RepositoryVersion GET response (when sync task complete):

{
    "base_version": null,
    "content_summary": {
        "added": {
            "rpm.advisory": {
                "count": 4,
                "href": "/pulp/api/v3/content/rpm/advisories/?repository_version_added=/pulp/api/v3/repositories/rpm/rpm/a02ace53-d490-458d-8b93-604fbcd23a9c/versions/1/"
            },
            "rpm.package": {
                "count": 35,
                "href": "/pulp/api/v3/content/rpm/packages/?repository_version_added=/pulp/api/v3/repositories/rpm/rpm/a02ace53-d490-458d-8b93-604fbcd23a9c/versions/1/"
            },
            "rpm.packagecategory": {
                "count": 1,
                "href": "/pulp/api/v3/content/rpm/packagecategories/?repository_version_added=/pulp/api/v3/repositories/rpm/rpm/a02ace53-d490-458d-8b93-604fbcd23a9c/versions/1/"
            },
            "rpm.packagegroup": {
                "count": 2,
                "href": "/pulp/api/v3/content/rpm/packagegroups/?repository_version_added=/pulp/api/v3/repositories/rpm/rpm/a02ace53-d490-458d-8b93-604fbcd23a9c/versions/1/"
            },
            "rpm.packagelangpacks": {
                "count": 1,
                "href": "/pulp/api/v3/content/rpm/packagelangpacks/?repository_version_added=/pulp/api/v3/repositories/rpm/rpm/a02ace53-d490-458d-8b93-604fbcd23a9c/versions/1/"
            }
        },
        "present": {
            "rpm.advisory": {
                "count": 4,
                "href": "/pulp/api/v3/content/rpm/advisories/?repository_version=/pulp/api/v3/repositories/rpm/rpm/a02ace53-d490-458d-8b93-604fbcd23a9c/versions/1/"
            },
            "rpm.package": {
                "count": 35,
                "href": "/pulp/api/v3/content/rpm/packages/?repository_version=/pulp/api/v3/repositories/rpm/rpm/a02ace53-d490-458d-8b93-604fbcd23a9c/versions/1/"
            },
            "rpm.packagecategory": {
                "count": 1,
                "href": "/pulp/api/v3/content/rpm/packagecategories/?repository_version=/pulp/api/v3/repositories/rpm/rpm/a02ace53-d490-458d-8b93-604fbcd23a9c/versions/1/"
            },
            "rpm.packagegroup": {
                "count": 2,
                "href": "/pulp/api/v3/content/rpm/packagegroups/?repository_version=/pulp/api/v3/repositories/rpm/rpm/a02ace53-d490-458d-8b93-604fbcd23a9c/versions/1/"
            },
            "rpm.packagelangpacks": {
                "count": 1,
                "href": "/pulp/api/v3/content/rpm/packagelangpacks/?repository_version=/pulp/api/v3/repositories/rpm/rpm/a02ace53-d490-458d-8b93-604fbcd23a9c/versions/1/"
            }
        },
        "removed": {}
    },
    "number": 1,
    "pulp_created": "2019-11-27T13:30:31.961788Z",
    "pulp_href": "/pulp/api/v3/repositories/rpm/rpm/a02ace53-d490-458d-8b93-604fbcd23a9c/versions/1/"
}

Create a Publication

A publication can only be created once a sync task completes. You can specify checksum algorithm with the following optional parameters:

  • metadata_checksum_type: affects all the repodata, including primary.xml, repomd.xml, etc.
  • package_checksum_type: affects package checksum type in all repo metadata files.
#!/usr/bin/env bash

# Create RPM publication
echo "Create a task to create a publication."
export TASK_URL=$(http POST $BASE_ADDR/pulp/api/v3/publications/rpm/rpm/ \
    repository=$REPO_HREF metadata_checksum_type=sha256 | jq -r '.task')

# Poll the task (here we use a function defined in docs/_scripts/base.sh)
wait_until_task_finished $BASE_ADDR$TASK_URL

# After the task is complete, it gives us a new publication
echo "Set PUBLICATION_HREF from finished task."
export PUBLICATION_HREF=$(http $BASE_ADDR$TASK_URL| jq -r '.created_resources | first')

echo "Inspecting Publication."
http $BASE_ADDR$PUBLICATION_HREF

Publication GET response (when task complete):

{
    "publisher": null,
    "pulp_created": "2019-11-27T13:30:36.006972Z",
    "pulp_href": "/pulp/api/v3/publications/rpm/rpm/c90316fc-bf2a-458a-93b8-d3d75614572f/",
    "repository": "/pulp/api/v3/repositories/rpm/rpm/a02ace53-d490-458d-8b93-604fbcd23a9c/",
    "repository_version": "/pulp/api/v3/repositories/rpm/rpm/a02ace53-d490-458d-8b93-604fbcd23a9c/versions/1/"
}

Create a Distribution for the Publication

#!/usr/bin/env bash

# Variables
if [ $# -eq 0 ]; then
  export BASE_PATH="foo"
else
  export BASE_PATH="$1"
fi

# Create RPM distribution for publication
export TASK_URL=$(http POST $BASE_ADDR/pulp/api/v3/distributions/rpm/rpm/ \
    publication=$PUBLICATION_HREF name="$BASE_PATH" base_path="$REPO_NAME" | jq -r '.task')

# Poll the task (here we use a function defined in docs/_scripts/base.sh)
wait_until_task_finished $BASE_ADDR$TASK_URL

# After the task is complete, it gives us a new distribution
echo "Set DISTRIBUTION_HREF from finished task."
export DISTRIBUTION_HREF=$(http $BASE_ADDR$TASK_URL| jq -r '.created_resources | first')

echo "Inspecting Distribution."
http $BASE_ADDR$DISTRIBUTION_HREF

Distribution GET response (when task complete):

{
    "base_path": "foo",
    "base_url": "http://pulp3-source-fedora30.pavels-macbook-pro.example.com/pulp/content/foo",
    "content_guard": null,
    "name": "baz",
    "publication": "/pulp/api/v3/publications/rpm/rpm/c90316fc-bf2a-458a-93b8-d3d75614572f/",
    "pulp_created": "2019-11-27T13:30:38.238857Z",
    "pulp_href": "/pulp/api/v3/distributions/rpm/rpm/c1166d2d-0832-4e90-85fd-e34e94e6a156/"
}