summaryrefslogtreecommitdiff
path: root/lib/libalpm
AgeCommit message (Collapse)Author
2011-07-21Convert package filelists to an array instead of linked listDan McGee
This accomplishes quite a few things with one rather invasive change. 1. Iteration is much more performant, due to a reduction in pointer chasing and linear item access. 2. Data structures are smaller- we no longer have the overhead of the linked list as the file struts are now laid out consecutively in memory. 3. Memory allocation has been massively reworked. Before, we would allocate three different pieces of memory per file item- the list struct, the file struct, and the copied filename. What this resulted in was massive fragmentation of memory when loading filelists since the memory allocator had to leave holes all over the place. The new situation here now removes the need for any list item allocation; allocates the file structs in contiguous memory (and reallocs as necessary), leaving only the strings as individually allocated. Tests using valgrind (massif) show some pretty significant memory reductions on the worst case `pacman -Ql > /dev/null` (366387 files on my machine): Before: Peak heap: 54,416,024 B Useful heap: 36,840,692 B Extra heap: 17,575,332 B After: Peak heap: 38,004,352 B Useful heap: 28,101,347 B Extra heap: 9,903,005 B Several small helper methods have been introduced, including a list to array conversion helper as well as a filelist merge sort that works directly on arrays. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-19Clean up my debug logger messDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-18Fix test suite when GPGME is disabledDan McGee
As noted by Allan, we failed pretty hard if gpgme was compiled out. With these changes, only sign001.py fails. This can/will be fixed later once we beef up the test suite with more signing tests anyway. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-18replace access() calls for debug info where applicableFlorian Pritz
Signed-off-by: Florian Pritz <bluewind@xinu.at> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-18signing.c: check if needed files are readableFlorian Pritz
If we can't read the keyring, gpgme will output confusing debug information and fail to verify the signature, so we should log some debug information. Signed-off-by: Florian Pritz <bluewind@xinu.at> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-18add _alpm_access() wrapperFlorian Pritz
This is a wrapper function for access() which logs some debug information and eases handling in case of split directory and filename. Signed-off-by: Florian Pritz <bluewind@xinu.at> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-18Fix compilation without gpgmeAllan McRae
Signed-off-by: Allan McRae <allan@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-18Handle removal of empty directories properlyDan McGee
This addresses FS#25141. We shouldn't remove every empty directory we come across during the removal process unless it is truly not known to any other package. This will prevent removal of essential directories such as '/var/lock/'. This is accomplished by first checking the empty/non-empty status of a directory, which was previously done implicitly by calling rmdir() and ignoring errors. We do this to avoid the next (new) check in most cases, which is to look at all local packages to see if the to-be-removed directory is present in another packages' filelist. If we do not find it anywhere, then we remove it, else we keep the file around. The pactest has been updated to test more cases, as well as finding a flaw in the original expected to fail case- we need separate DIR and FILE based EXIST rules. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-14Make alpm_db_set_pkgreason() arguments more saneDan McGee
This can only ever operate on the local database, and a local package at that. Change the function signature to take a handle and package object, add the relevant asserts, and ensure the frontend can detect the package not found condition when finding packages to pass to this method. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-14Allow fileconflict if unowned file moving into backup arrayDan McGee
The bulk of this commit is adding new tests to ensure the new behavior works without disrupting old behavior. This is a relatively sane maneuver when a package adds a conf file (e.g. '/etc/mercurial/hgrc') that was not previously in the package, but it is placed in the backup array. In essence, we can treat the existing file as having always been a part of the package and do our normal compare/install as pacnew logic checks. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-06Unify package removal codeDan McGee
This code duplication has always been a rather clumsy casuality of fixing some past upgrade issues. Unify the removal code across upgrade and remove operations into a new _alpm_remove_single_package() method wihch makes it very clear how we handle upgrade and remove differently, via several conditionals on newpkg. This commit highlights interesting behavior such as the fact that the implicit removal in every package upgrade never gets transaction events or progress callbacks. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05include util.h in rawstr.cDan McGee
Fixes "error: no previous prototype for '_alpm_raw_cmp' [-Werror=missing-prototypes]" warnings, and also prevents someone from getting the prototypes and functions out of sync. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05Merge remote-tracking branch 'dave/download'Dan McGee
2011-07-05absorb fileinfo struct into dload_payloadDave Reisner
This transitional struct becomes delicious noms for dload_payload. Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-07-05absorb some _alpm_download params into payload structDave Reisner
Restore some sanity to the number of arguments passed to _alpm_download and curl_download_internal. Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-07-05lib/dload: prevent large file attacksDave Reisner
This means creating a new struct which can pass more descriptive data from the back end sync functions to the downloader. In particular, we're interested in the download size read from the sync DB. When the remote server reports a size larger than this (via a content-length header), abort the transfer. In cases where the size is unknown, we set a hard upper limit of: * 25MiB for a sync DB * 16KiB for a signature For reference, 25MiB is more than twice the size of all of the current binary repos (with files) combined, and 16KiB is a truly gargantuan signature. Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-07-05dload: handle irregular URLsDave Reisner
URLs might end with a slash and follow redirects, or could be a generated by a script such as /getpkg.php?id=12345. In both cases, we may have a better filename that we can write to, taken from either content-disposition header, or the effective URL. Specific to the first case, we write to a temporary file of the format 'alpmtmp.XXXXXX', where XXXXXX is randomized by mkstemp(3). Since this is a randomly generated file, we cannot support resuming and the file is unlinked in the event of an interrupt. We also run into the possibility of changing out the filename from under alpm on a -U operation, so callers of _alpm_download can optionally pass a pointer to a *char to be filled in by curl_download_internal with the actual filename we wrote to. Any sync operation will pass a NULL pointer here, as we rely on specific names for packages from a mirror. Fixes FS#22645. Signed-off-by: Dave Reisner <d@falconindy.com>
2011-07-05Remove most usages of strncmp()Dan McGee
The supposed safety blanket of this function is better handled by explicit length checking and usages of strlen() on known NULL-terminated strings rather than hoping things fit in a buffer. We also have no need to fully fill a PATH_MAX length variable with NULLs every time as long as a single terminating byte is there. Remove usages of it by using strcpy() or memcpy() as appropriate, after doing length checks via strlen(). Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05dload: rearrange code to avoid extra cpp blockDave Reisner
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-07-05lib/rawstr: borrow raw string functions from curlDave Reisner
We'll need these functions to do locale agnostic and case insensitive string comparisons. Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-07-05Simplify alpm_list_previousAllan McRae
We can readily detect the first node in a list by checking if node->prev->next is NULL. So there is no need to pass the head of the list to this function and its prototype now looks like all the other item accessors. Signed-off-by: Allan McRae <allan@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05Remove alpm_list_firstAllan McRae
The only thing this accessor did was remove the const qualifier given our entire list implementation requires passing around the head anyway. Signed-off-by: Allan McRae <allan@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05Allow invalid sync DBs to be returned by the libraryDan McGee
They are placeholders, but important for things like trying to re-sync a database missing a signature. By using the alpm_db_validity() method at the right time, a client can take the appropriate action with these invalid databases as necessary. In pacman's case, we disallow just about anything that involves looking at a sync database outside of an '-Sy' operation (although we do check the validity immediately after). A few operations are still permitted- '-Q' ops that don't touch sync databases as well as '-R'. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05Allow frontend access to signature verification informationDan McGee
Show output in -Qip for each package signature, which includes the UID string from the key ("Joe User <joe@example.com>") and the validity of said key. Example output: Signatures : Valid signature from "Dan McGee <dpmcgee@gmail.com>" Unknown signature from "<Key Unknown>" Invalid signature from "Dan McGee <dpmcgee@gmail.com>" Also add a backend alpm_sigresult_cleanup() function since memory allocation took place on this object, and we need some way of freeing it. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05Correctly check the GPG error codesDan McGee
The error code is in fact a bitmask value of an error code and an error source, so use the proper function to get only the relevant bits. For the no error case, this shouldn't ever matter, but it bit me when I was trying to compare the error code to other values and wondered why it wasn't working, so set a good example. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05signing: move to new signing verification and return schemeDan McGee
This gives us more granularity than the former Never/Optional/Always trifecta. The frontend still uses these values temporarily but that will be changed in a future patch. * Use 'siglevel' consistenly in method names, 'level' as variable name * The level becomes an enum bitmask value for flexibility * Signature check methods now return a array of status codes rather than a simple integer success/failure value. This allows callers to determine whether things such as an unknown signature are valid. * Specific signature error codes mostly disappear in favor of the above returned status code; pm_errno is now set only to PKG_INVALID_SIG or DB_INVALID_SIG as appropriate. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03Merge remote-tracking branch 'allan/ALPM'Dan McGee
Conflicts: lib/libalpm/be_local.c lib/libalpm/be_package.c lib/libalpm/conflict.c lib/libalpm/diskspace.c lib/libalpm/dload.c lib/libalpm/remove.c
2011-07-03conflict code tweaks and cleanupsDan McGee
We can take advantage of a few things on our new and improved filelist in this code. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03diskspace: remove all libarchive usageDan McGee
Now that the filelists capture mode and size information, we can read the data from there and prevent having to loop through and uncompress every archive to check required diskspace usage. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03Move alpm filelists to a struct objectDan McGee
This allows us to capture size and mode data when building filelists from package files. Future patches will take advantage of this newly available information, and frontends can use it as well. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03Fix debug logger without a newlineDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03Do not replicate files list when removing packagesAllan McRae
This saves replicating the potentially large list of files in a package that is being removed. Signed-off-by: Allan McRae <allan@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03Add alpm_list_previous methodAllan McRae
Helper function to get the previous item in a list Signed-off-by: Allan McRae <allan@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03checkdeps: remove unnecessary list join and copyDan McGee
We can just perform the same search operation on both lists. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03be_local: use macros in database loading similar to be_syncDan McGee
This removes some of the repetition in the code for reading and parsing database file lines. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-03be_sync: make READ_NEXT() a no-arg macroDan McGee
We passed in 'line', but not 'buf.line'. In addition, the macros building off of READ_NEXT() assume variable names anyway. Since we only use these macros in one function, might as well simplify them. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-02Prefix _alpm_errno_t members with ALPMAllan McRae
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02Prefix alpm_transprog_t members with ALPMAllan McRae
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02Prefix alpm_transconv_t members with ALPMAllan McRae
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02Prefix alpm_transevt_t members with ALPMAllan McRae
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02Prefix alpm_transflag_t members with ALPMAllan McRae
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02Prefix alpm_loglevel_t members with ALPMAllan McRae
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02Prefix alpm_fileconflicttype_t members with ALPMAllan McRae
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02Prefix alpm_depmod_t members with ALPMAllan McRae
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-07-02Prefix alpm_pkgreason_t members with ALPMAllan McRae
Signed-off-by: Allan McRae <allan@archlinux.org>
2011-06-30signing: check validity of all available signaturesDan McGee
Change the check into a loop over all signatures present and returned by GPGME. Also modify the return values and checks slightly now that I know a little bit more about what type of values are returned. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30Make alpm_db_get_sigverify_level() publicDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30Add an alpm_db_get_valid() public functionDan McGee
This allows one to check if a database is valid or invalid. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30Make local_db_read() private to the local backendDan McGee
There is little need to expose the guts of this function even within the library. Make it static in be_local.c, and clean up a few other things since we know exactly where it is being called from: * Remove unnecessary origin checks in _cache_get_*() methods- if you are calling a cache method your package type will be correct. * Remove sanity checks within local_db_read() itself- packages will always have a name and version if they get this far, and the package object will never be NULL either. The one case calling this from outside the backend was in add.c, where we forced a full load of a package before we duplicated it. Move this concern elsewhere and have pkg_dup() always force a full package load via a new force_load() function on the operations callback struct. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-30Fix a few warnings pointed out via clang scan-buildDan McGee
Some of these are legit (the backup hash NULL checks), while others are either extemely unlikely or just impossible for the static code analysis to prove, but are worth adding anyway because they have little overhead. Signed-off-by: Dan McGee <dan@archlinux.org>