summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2020-06-29.gitignore: devtools -> devtools32Erich Eckner
2020-06-29Makefile: devtools -> devtools32 in dist and upload targetsErich Eckner
2020-06-29Revert "Remove i686 support"Luke Shumaker
This reverts commit 7259e7def07a5f6ee04a34db61a87361ad0b5ac7, except for commitpkg.in
2020-06-29generate separate config file for i686 and x86_64Erich Eckner
2020-06-29mkarchroot.in: copy host's /etc/hosts into the chrootErich Eckner
2020-06-29arch-nspawn should use the correct pacman config fileErich Eckner
If arch-nspawn is called with -C, pacman inside the chroot will use the provided configuration file. This should also be the case for $pacconf_cmd and pacman outside the chroot. If arch-nspawn is called without -C, pacman inside the chroot will use $workdir/etc/pacman.conf -- again, $pacconf_cmd and pacman outside the chroot should use that, too. So lets just set $pac_conf in that case. For example, Arch Linux 32 provides separate pacman configurations inside /usr/share/devtools which use /etc/pacman.d/mirrorlist32 as mirrorlist for their build commands (extra-i686-build, etc.). This way, we can build i686 and x86_64 packages on the same x86_64 host with very minimal changes to devtools.
2020-06-29use `read -r var < "$file"` instead of `var=$(cat "$file")`Erich Eckner
2020-06-29allow to call setarch with a different value than $CARCHErich Eckner
Introduce setarch-aliases.d/ which gets installed inside /usr/share/devtools. This allows to assign aliases which map one CARCH to a different name which gets provided as argument for setarch. This is necessary on archlinuxarm ("armv6h" -> "armv6l", "armv7h" -> "armv7l") and allows for more fine-grained architectures (e.g. archlinux32 has "i686" and "pentium4", which differ in the required cpu capabilities).
2020-04-08allow PKGEXT to be set from the environmentErich Eckner
2020-04-07Version 2020040720200407Levente Polyak
2020-03-09offload-build: retrieve the PKGBUILD in case makepkg changes it via pkgverLevente Polyak
Building a package may change the PKGBUILD during update_pkgver. Let's retrieve the PKGBUILD after building to ensure we have the very same file as the one we used to build the package. Otherwise this may lead to the inability to distribute the package during commitpkg in case the expected and the actual hashsum mismatch.
2020-03-09makerepropkg: prioritize downloading .zst packages over legacy formatLevente Polyak
First try a .zst location before falling back to legacy variants. This should slightly speed up downloading of dependencies, especially over time as .zst packages are or will be the dominant format. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2020-02-27arch-nspawn: fix up host_mirrors cachedir handlingEli Schwartz
This was incorrectly implemented in commit 00671765296ad598f392df7a542dbfca48a6e7b7, which added the host_mirrors root directory as a cachedir, when we actually want to use the pool/* subdirectories (the same ones installed on the build server's /etc/pacman.conf). Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2020-02-27makechrootpkg: use the chroot database to find checkpkg packagesEli Schwartz
We don't want to check against the current version known to the host system, because that will be incorrect in a wide variety of situations, including: - the build host hasn't done a full system upgrade yet - we're building against staging, and want to see the delta between different staging versions - we're building against extra, but the host runs testing which carries changes we don't want to visualize right now - the chroot has a configured database not available to the host, and the package is only available there Essentially, it's rarely 100% correct to run checkpkg on the host, but we already have a database we *know* is correct, and that is the one we just built the package against. So let's use that. This also fixes a bug in the current logic, where in order to try downloading fresh databases, we work in a non-cached temporary working database to download the package files, but then let checkpkg default to comparing packages in the system database. Since we are explicitly trying to compare against packages that differ from the host's pacman database, we need to pass the package files as options to checkpkg, using the additional modes added in commit c14338c0fe71a74f5e56b4f3af7c548fe0928e15 Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2020-02-27checkpkg: fix support for http:// url comparisonsEli Schwartz
Broken in commit c14338c0fe71a74f5e56b4f3af7c548fe0928e15. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2020-02-27zsh-completion: update for new makerepropkg optionsEli Schwartz
Thanks to anthraxx for the guidance. Original-patch-by: Levente Polyak <anthraxx@archlinux.org> Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2020-02-27makerepropkg: support checking multiple split packagesEli Schwartz
By specifying multiple package files, we assume they are all from the same PKGBUILD, and try to check them all against the produced artifacts. Since the buildinfo should be comparable for all of them, we simply use the first one passed on the command line. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2020-02-27makerepropkg: add support to check unreproducible packages using diffoscopeEli Schwartz
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2020-02-27makerepropkg: fix wonky indentEli Schwartz
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2020-02-13Version 2020021320200213Levente Polyak
2020-02-12offload-build: handle user specific makepkg.conf on the remote hostLevente Polyak
This ensures we take user specific config values for PKGDEST into account when printing the package list. This is required as devtools archbuild_cmd puts packages potentially into the user defined PKGDEST which the package list would otherwise miss.
2020-02-11offload-build: take makepkg.conf from devtools for appropriate packagelistLevente Polyak
This fixes an issue with the usage of makepkg --packagelist to get the produced artifacts filenames according to the PKGEXT used in devtools' makepkg.conf instead of the one defined in pacman. One goal we want to preserve is that devtools configuration should be self contained and not require any editing of non owned files like the host /etc/makepkg.conf to produce expected results. Additionally modify the archbuild_cmd override for multilib builds to use an independent variable and not fiddle with the actual arch variable to select the appropriate cmd.
2020-02-10offload-build: convert to in-prog so we can perform pre-processingLevente Polyak
2020-02-10makerepropkg: take makepkg.conf from devtools pkgdatadir locationLevente Polyak
This ensures we use the same configuration for reproducing packages as we use for building them via devtools. One example of why we care about this are the COMPRESS* settings that may differ from the guest's pacman shipped makepkg.conf that affect the reproducibility of packages.
2019-12-27makerepropkg: correctly reproduce a pkgfile with any compression typeEli Schwartz
We don't want the default PKGEXT in the current version of devtools, we want the PKGEXT we *know* the input file used. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-12-27Version 2019122720191227Levente Polyak
2019-12-27makerepopkg: fix zstd extension (.zstd -> .zst)Robin Broda
2019-12-27makepkg.conf: change default compression method to zstdRobin Broda
2019-12-12Version 2019121220191212Levente Polyak
2019-12-08lib: support arbitrary compressed package filesLevente Polyak
We need to modify the matching of valid package files to support formats like zstd. Let's try to use an eager approach instead of a simple whitelist in order to be functional for arbitrary formats that may be introduced in the future without the need to adjust any code. Allow any single fragment word as compression type but filter out known non-package content like signature files. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-12-05offload-build: return failure if no package files were foundEli Schwartz
This means that the remote command died at some stage earlier than the printing of created files. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
2019-12-05makechrootpkg: sync databases for checkpkg off-siteLevente Polyak
Use pacman's --dbpath feature to sync fresh databases inside an isolated location and split up the database sync and package location calls to remove the need of weird grep calls. It isn't nice of makechrootpkg to modify the host database state just by building packages. No foreign program shall automatically modify the host database other than by the explicit will of a system maintainer, which is the major reason this changes get incorporated. However, there is certain indoctrinated believe that using -Sy is the prime evil. In fact it has been declared as a social rule to a technical problem of not getting into potential partial upgrade states. This is not a proper loophole less solution as there are multiple ways and use cases that lead to such a state, like aborting a -Syu on the prompt for whatever reason, what really matters is that it is not a technically bullet proof solution to solve the problem. Databases shall have the freedom to be as up to date as databases or their owner wishes, allowing querying on latest database state without fear. The only loophole-less contract that _really_ is from importance is always using -Su instead of plain -S to install packages. Installing packages is what actually brings one into a potential partial upgrade state and by using -Su an outstanding upgrade is forced when installing a new package. This properly solves all edge cases in a technical manner instead of declaring people who abort the prompt of -Syu to be the problem. In fact, using this simple contract allows whatever system maintenance workflow a host owner wants to follow, which may still be to always use -Syu and deal with system upgrades explicitly instead of the time when installing new packages, but the -Su contract is the real safe guard to guarantee no edge case can ever slip in. This magically also opens up the freedom to people who wish to use -Sy to simply query on up to date data as the currently indoctrinated "never do -Sy" stone plates not only are not rock solid in technical terms but also make certain use cases simply impossible and hence cripple the functionality without at the very least being fully loophole free. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-11-30makechrootpkg: double quote array expansions for checkpkg locationsLevente Polyak
Avoid re-splitting remotepkg elements used for checkpkg conditions.
2019-11-30Version 2019113020191130Levente Polyak
2019-11-30makechrootpkg: sync database for checkpkg to avoid nonexistent targetsLevente Polyak
For build servers or similar infrastructure its relatively common to not sync/update the database regularly. This leads to problems properly running checkpkg duo to nonexistent target files that we try to download. As building on build servers is a very common use case, lets ensure we sync the local database before trying to resolve the package locations.
2019-11-30makechrootpkg: check local pkg versions before downloading for checkpkgLevente Polyak
Avoid always trying to download and output the according message. Add checks for packages either not being available in the repo or all variants have up to date versions stored in the local cache.
2019-11-30completion: add makerepropkg zsh completionLevente Polyak
2019-11-30completion: reflect new -U mkarchchroot optionLevente Polyak
2019-11-30lib/common.sh: do not use colors when running on a dumb terminalIvy Foster
2019-11-30doc: add manpage for the new makerepropkg toolEli Schwartz
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
2019-11-30makerepropkg: add new program to try to reproducibly build a packageEli Schwartz
This attempts to recreate a package that was probably created using makechrootpkg, and see if it conforms to the https://reproducible-builds.org/ specification. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
2019-11-30mkarchroot: support wrapping pacstrap -UEli Schwartz
Needed to support reproducible builds. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
2019-11-30archbuild: use better umaskEli Schwartz
In commit 40a90e2cab479cc64903a62b42eb617a8a7e5842 we tried to protect against system umasks resulting in unreadable chroots. However, we tried to do this in a targeted manner due to not wanting to fiddle with permissions for user-owned files. Unfortuantely, mkdir -p -m755 does not actually work that way -- the parent directory is created with broken permissions. We need umask. Run umask and mkdir in a subshell to prevent leakage. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
2019-11-30commitpkg: clearly delineate the commit message template and the rationaleEli Schwartz
In commit 75d23eec942e7160108ee194894b6b83ed3045d5 we moved to include commitpkg arguments as the first line of the svn commit message, but we simply dumped the result after the version number without separating the two, increasing the cognitive burden of parsing the rationale. Since the whole point of the change was to make it easier to see what happened when using git log --oneline (reducing the cognitive burden of parsing 'pretty' output with author/date info), it makes sense to also delineate the reason correctly. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
2019-11-30arch-nspawn: rely on deduplication to avoid making multiple host_mirror mountsEli Schwartz
Instead of comparing exact mirror urls to see if they are in host_mirrors in order to "skip" the official mirrors Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
2019-11-30offload-build: remove empty src dir when SRCDEST is setLevente Polyak
Right now there is a bug in makepkg that leaves back an empty src directory if SRCDEST is set. This is purely cosmetic, but lets just politely try to rmdir it and fail silently if its empty or non-existent. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-11-30offload-build: catch more signals to be sure SRCPKGDEST gets cleanedLevente Polyak
It may be not enough to just listen on EXIT depending on the shell used so lets make sure we clean up SRCPKGDEST by listening to more sigs. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-11-07Makefile: sort program namesEli Schwartz
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-11-07use libmakepkg to find and use variables in makepkg.confEli Schwartz
- drop homebrew function in makechrootpkg - use better mock to find invoking user's $HOME - make offload-build respect makepkg.conf to determine where to sync files, matching the behavior of makechrootpkg Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-11-07arch-nspawn: simplify cachedir/host_mirror parsingEli Schwartz
consolidate logic flows in the same area for parsing and building arrays. Don't bother having a special function just to build the mount_args array, since we now use the same handling for adding any cachedir (including host mirrors) to the mount arguments, this becomes a trivial for loop -- and it really did not need to be delayed until after the sanity check, anyway. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>