summaryrefslogtreecommitdiff
path: root/lib/libalpm/be_files.c
AgeCommit message (Collapse)Author
2009-11-15download: major refactor to address lingering issuesDan McGee
Sorry for this being such a huge patch, but I believe it is necessary for quite a few reasons which I will attempt to explain herein. I've been mulling this over for a while, but wasn't super happy with making the download interface more complex. Instead, if we carefully order things in the internal download code, we can actually make the interface simpler. 1. FS#15657 - This involves `name.db.tar.gz.part` files being left around the filesystem, and then causing all sorts of issues when someone attempts to rerun the operation they canceled. We need to ensure that if we resume a download, we are resuming it on exactly the same file; if we cannot be almost postive of that then we need to start over. 2. http://www.mail-archive.com/pacman-dev@archlinux.org/msg03536.html - Here we have a lighttpd bug to ruin the day. If we send both a Range: header and If-Modified-Since: header across the wire in a GET request, lighttpd doesn't do what we want in several cases. If the file hadn't been modified, it returns a '304 Not Modified' instead of a '206 Partial Content'. We need to do a stat (e.g. HEAD in HTTP terms) operation here, and the proceed accordingly based off the values we get back from it. 3. The mtime stuff was rather ugly, and relied on the called function to write back to a passed in reference, which isn't the greatest. Instead, use the power of the filesystem to contain this info. Every file downloaded internally is now carefully timestamped with the remote file time. This should allow the resume logic to work. In order to guarantee this, we need to implement a signal handler that catches interrupts, notifies the running code, and causes it to set the mtimes on the file. It then rethrows the signal so the pacman signal handler (or any frontend) works as expected. 4. We did a lot of funky stuff in trying to track the DB last modified time. It is a lot easier to just keep the downloaded DB file around and track the time on that rather than in a funky dot file. It also kills a lot of code. 5. For GPG verification of the databases down the road, we are going to need the DB file around for at least a short bit of time anyway, so this gets us closer to that. Signed-off-by: Dan McGee <dan@archlinux.org> [Xav: fixed printf with off_t] Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
2009-10-19Size handling was changed in fgets() functionsLaszlo Papp
Pacman's fgets function in the API used hardcoded numbers to identify the size. This is not good practice, so replace them with sizeof handling. Signed-off-by: Laszlo Papp <djszapi@archlinux.us> Signed-off-by: Dan McGee <dan@archlinux.org>
2009-10-11Rework the alpm_unpack functionsXavier Chantry
Add support to extract a list of entries Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2009-10-11cygwin fix : use unsigned char for ctype functionXavier Chantry
See http://www.nabble.com/-PATCH-RFA--Distinguish-between-EOF-and-character-with-value-0xff-td23161772.html#a23188494 cygwin 1.7 actually displays a warning when using signed char with the ctype function, so that compilation fails when using -Wall -Werror. So we just cast all arguments to unsigned char. Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2009-09-20Fully implement database lazy loadingDan McGee
Commit 34e1413d75 attempted to implement lazy loading of package databases. Although it took care of my main complaint (creating the database directory if it didn't exist), it didn't allow sync repos to be registered before alpm_option_set_dbpath() had been called. With this patch, we no longer compute the individual repository DB paths until necessary, allowing full lazy loading to work as intended, and allowing us to drop the extra setlibpath() calls from the frontend. This allows the changes introduced in a2cd48960 (but later reverted) to be added back in again. Signed-off-by: Dan McGee <dan@archlinux.org>
2009-09-20Merge branch 'maint'Dan McGee
Message updates made this one a bit messy, but nothing too bad. Conflicts: lib/libalpm/add.c lib/libalpm/remove.c
2009-09-16String improvementsXavier Chantry
Add more untranslated strings, improve consistency, etc. Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2009-09-08Remove transaction typeXavier Chantry
This basically started with this change : /* Transaction */ struct __pmtrans_t { - pmtranstype_t type; pmtransflag_t flags; pmtransstate_t state; - alpm_list_t *packages; /* list of (pmpkg_t *) */ + alpm_list_t *add; /* list of (pmpkg_t *) */ + alpm_list_t *remove; /* list of (pmpkg_t *) */ And then I have to modify all the code accordingly.
2009-05-11Fix non-C style comments in libalpmGerardo Exequiel Pozzi
Signed-off-by: Gerardo Exequiel Pozzi <vmlinuz386@yahoo.com.ar> Signed-off-by: Dan McGee <dan@archlinux.org>
2009-04-11Add detail documentation for alpm_db_update()Sebastian Nowicki
This adds alpm_db_update() to the alpm_databases Doxygen group. The function is described in more detail and a code example is given. Signed-off-by: Dan McGee <dan@archlinux.org>
2009-03-15be_files.c : use %jd instead of %ju for intmax_tXavier Chantry
Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2009-01-20Delay the creation of local and sync db dir.Xavier Chantry
We don't need to create the directories when local or sync dbs are registered. For example, if a sync db does not exist, we cannot even do "pacman -Q" as an user. Instead, we can create the local db if needed during the db_prepare operation, and sync dbs on db_update. Also remove some more useless abstractions in db_update and switch to a much more efficient way to remove a sync db : rm -rf. Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
2009-01-20Remove some db abstraction crap.Xavier Chantry
These db_open and db_close looked quite useless. And they caused the db directory to be opened on a simple registering of a database. This is totally unneeded, this opening can be delayed to when we actually need it. Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
2009-01-12Rename alpm_get_md5sum to alpm_compute_md5sum and alpm_dep_get_string to ↵Nagy Gabor
alpm_dep_compute_string This patch introduces the following function name convention: _compute_ in function name: the return value must be freed. _get_ in function name: the return value must not be freed. Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu> Signed-off-by: Dan McGee <dan@archlinux.org>
2009-01-02Separate local db directory creation and db writeAllan McRae
Changelogs and install files were getting extracted into the local db folder before it was manually created. This created issues for uses with 0077 umasks and was highlighted with the new sudo handling of umasks (FS#12263). This moves the local db creation to its own function which is called before the start of package archive extraction. Also, added a check that the folder is actually created. Signed-off-by: Allan McRae <allan@archlinux.org> [Dan: rename to _alpm_db_prepare()] Signed-off-by: Dan McGee <dan@archlinux.org>
2008-11-30Add regex to delta code so we don't segfault when reading lineDan McGee
If the delta line doesn't match our regex, we won't go and process it, possibly walking off the end of the string. Signed-off-by: Dan McGee <dan@archlinux.org>
2008-11-17Minor code cleanupsDan McGee
Mostly noticed when compiling libalpm/pacman with ICC. Signed-off-by: Dan McGee <dan@archlinux.org>
2008-06-15Use access() instead of stat() when possibleDan McGee
We were using the stat() system call in quite a few places when we didn't actually need anything the stat struct returned- we were simply checking for file existence. access() will be more efficient in those cases. Before (strace pacman -Ss pacman): % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 33.16 0.005987 0 19016 stat64 After: % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 34.85 0.003863 0 12633 1 access 7.95 0.000881 0 6391 7 stat64 Signed-off-by: Dan McGee <dan@archlinux.org>
2008-06-04Get rid of double / in database paths.Xavier Chantry
Errors like the following one happen regularly (for unknown reasons...) : error: could not open file /var/lib/pacman/local//glibc-2.7-9/depends: No such file or directory Anyway, every time an user reported an error like that, it always seemed like he thought the error was caused by the double /, which is obviously wrong. Since db->path always include a trailing /, there is no need to add one when concatenating paths in be_files.c or add.c. Additionally, some static strings were switched to dynamic. And the computation of the "dbpath"/"pkgname"-"pkgversion" was refactored in db_read, db_write and db_remove with a get_pkgpath static function. Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2008-06-04Use correct C type for file sizesDan McGee
We have been using unsigned long as a file size type for a while, which works but isn't quite correct and could easily break. Worse was probably our use of int in the download callback functions, which could be restrictive for packages > 2GB in size. Switch all file size variables to use off_t, which is the preferred type for file sizes. Note that at least on Linux, all applications compiled against libalpm must now be sure to use large file support, where _FILE_OFFSET_BITS is defined to be 64 or there will be some weird issues that crop up. Signed-off-by: Dan McGee <dan@archlinux.org>
2008-05-29be_files : stores REPLACES and FORCE in desc.Xavier Chantry
repo-add and db_read both assume that REPLACES and FORCE fields are in the desc file, so do that for db_write as well (instead of depends file). Note that db_write is currently only used on the local database. And the only purpose of replaces and force in local database is for information purpose (available on -Qi operations). So this is not a big problem. Ref: http://www.archlinux.org/pipermail/pacman-dev/2008-May/011859.html Acked-by: Dan McGee <dan@archlinux.org> Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2008-05-29Add some verbosity to _alpm_db_read.Nagy Gabor
Name and version are computed from "/var/lib/pacman/..." pathname. And the %NAME% and %VERSION% fields from the desc file were not even read. So now, when we read the desc file, we make sure the %NAME% and %VERSION% fields are consistent. Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu> Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2008-05-13Simplify _alpm_pkg_new()Dan McGee
Any real call of this function doesn't specify a name or version ahead of time, so just kill that functionality off. Now to remove those dummy packages... Signed-off-by: Dan McGee <dan@archlinux.org>
2008-05-13Fix rewinddir regression by cleaning up db_scanXavier Chantry
Commit 046003844739416ff6d168dd2dec76490adb0727 caused a regression when rereading the pkgcache after updating the on-disk databases. A rewinddir call was errantly removed. Instead of replacing the call to rewindir, clean up this whole mess. db_scan is used only once and with target == NULL so there was actually half the code of db_scan which was unused. This is gone now and replaced by a single new db_populate function. Dan: add_sorted ended up being 3x slower than one msort at the end, so I changed back to that. I also made one pointer variable const and merged this whole patch with my original fix for the rewinddir issue. Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2008-05-11Remove some useless abstraction and start db cleanupDan McGee
We have some useless abstractions like an alpm_db_rewind function. I've read somewhere that readdir() was the worst filesystem function call invented, and what do we do? Add a wrapper around it. Kill this abstraction and move some other things into be_files that should be there anyway because they are so tied to how a files backend works. Signed-off-by: Dan McGee <dan@archlinux.org>
2008-05-08be_files.c: PATH_MAX cleanupDan McGee
Most of these are not easy to remove, but I could kill the ones in the two lastupdate functions. Signed-off-by: Dan McGee <dan@archlinux.org>
2008-04-06Remove unnecessary header file, move one macro to util.cDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2008-01-21Move the deptest code from frontend to backend.Chantry Xavier
The deptest code (pacman -T) used by makepkg was mostly in the frontend. There were 2 drawbacks: 1) the public splitdep function returns a pmdepend_t struct, but the _alpm_dep_free function for freeing it is private. So there was a memleak. 2) there is a helper in the backend (satisfycmp in deps.c) which makes this function much easier. So this adds a new public alpm_deptest in libalpm/deps.c, which cleans pacman_deptest in pacman/deptest.c a lot. Besides, alpm_splitdep was made private, because the frontend no longer requires it, and _alpm_dep_free is also private. Finally the deptest001 pactest was extended. Signed-off-by: Chantry Xavier <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2008-01-21Convert deltas to dynamic allocationDan McGee
Another elimination of a static length structure in libalpm. Should result in a little more memory saved during execution of packages with lots of deltas attached. Signed-off-by: Dan McGee <dan@archlinux.org>
2008-01-12Store replaces in the local DB tooAaron Griffin
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com> [Dan: might as well store 'force' too] Signed-off-by: Dan McGee <dan@archlinux.org>
2008-01-11Use dynamic string allocation in package structuresDan McGee
This also affects all structures with static strings, such as depmiss, conflict, etc. This should help a lot with memory usage, and hopefully make things a bit more "idiot proof". Currently our pactest pass/fail rate is identical before and after this patch. This is not to say it is a perfect patch- I have yet to pull valgrind out. However, this should be quite safe to use in all situations from here on out, and we can start plugging the memleaks. Original-work-by: Aaron Griffin <aaronmgriffin@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2008-01-01Remove gettext calls from debug-level messagesDan McGee
These used FUNCTION output level and not DEBUG, so I didn't catch them way back when I removed those gettext calls. Remove them now (which exposed a nice little memory access error elsewhere in the code). This should have a slight speedup effect on the code too as we no longer have to make the gettext call even when these messages aren't printed. Signed-off-by: Dan McGee <dan@archlinux.org>
2007-12-10Update GNU GPL boilerplate and copyright datesDan McGee
Update the GPL boilerplate to direct people to the GNU website for a copy of the license, as well as bump all of Judd's copyrights to 2007. Signed-off-by: Dan McGee <dan@archlinux.org>
2007-11-20Add missing header includes for setlocaleDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2007-11-16War on whitespaceDan McGee
Run the kernel's cleanfile script on all of our source files. Signed-off-by: Dan McGee <dan@archlinux.org>
2007-11-16libalpm: simplify sync db lastupdateDan McGee
Legacy code is hitting the trash here. Remove unnecessary _alpm_time2string time storage abstraction in favor of just writing the time_t value to the disk. The only drawback is that everyone's sync DBs will have to be updated at least once so that the lastupdate values are stored right. :) Signed-off-by: Dan McGee <dan@archlinux.org>
2007-11-14Move alpm_splitdep usage to db_readDan McGee
Holy inefficient batman! For a pacman -Qt operation (when we are using compute_requiredby and not database entries), splitdep was being called ~1.3 million times on my local database. By splitting when we read the DB, we drop this number to around 1700 and save a LOT of time in doing so (a 5x increase in pacman -Qt speed here). Note that the depends alpm_list_t in the package struct is no longer a string list, but a list of pmdepent_t objects. Signed-off-by: Dan McGee <dan@archlinux.org>
2007-11-14Remove REQUIREDBY usage from libalpmDan McGee
Instead of using the often-busted REQUIREDBY entries in the pacman database, compute them each time they are required. This should help many things: 1. Simplify the codebase 2. Prevent future database corruption 3. Ensure when we do use requiredby, it is always correct 4. Shrink the pmpkg_t memory overhead Signed-off-by: Dan McGee <dan@archlinux.org>
2007-11-13Fix display of -Qip output when a package file is givenDan McGee
Too many fields were being shown on -Qip output, and sizes were not always correct (-Qi and -Qip output on the same package did not agree). Signed-off-by: Dan McGee <dan@archlinux.org>
2007-11-08Fix some issues with localized dates/epoch usageDan McGee
Commit 47622eef4dd8fd86a0aa0e3ebdb7b33f7c9d6804 introduced localized times in the metadata by way of storing the UNIX epoch value instead of a hard coded date string. However, it missed a few things: * If we weren't in the C/POSIX/en_US locale, the date parsing would fail as it tried to use the abbreviations of the locale being used. Fix this by switching the LC_TIME value before we parse a date. * We used ctime to print the date value, which is always the C locale string. Instead, use strftime to print a localized date string. Signed-off-by: Dan McGee <dan@archlinux.org>
2007-10-23Print and typecast time_t correctlyDan McGee
The recommended C99 way to print the value of a time_t is to cast it to uintmax_t or intmax_t. Do this to ensure compatability with all platforms. Signed-off-by: Dan McGee <dan@archlinux.org>
2007-10-19Add pmdelta_t structure and functions to libalpm.Nathan Jones
Signed-off-by: Nathan Jones <nathanj@insightbb.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2007-09-28libalpm/be_files: Fix outdated code setting path for lastupdate fileDan McGee
The code for setting the lastupdate file path still referenced both the root and dbpath, when we should only be using dbpath. Fix it. Signed-off-by: Dan McGee <dan@archlinux.org>
2007-09-28Support for localized times in metadataAaron Griffin
Packages and DBs now support using the UNIX epoch (seconds since Jan 1, 1970) for use in builddate and installdate. This will only affect newly built packages. Old existing packages with the text format are still supported, but this is deprecated. In the case of removal of text time support, this code will fail gracefully, returning the start of the epoch for broken packages. Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2007-09-28Remove package name dependency from libalpmDan McGee
Previously, package names must match a specified scheme or they will cause pacman add operations to fail. This is not a very intelligent or necessary way to act, so remove the dependency on the name of the package to be installed and read all relevant information from the metadata instead. This does have one causality to be addressed later- pacman cache cleaning functionality, which has never been phenomenal, just lost most capability. Signed-off-by: Dan McGee <dan@archlinux.org>
2007-09-25Preliminary support for optdependsDan McGee
Add some alpm functions for getting optdepends, have makepkg include them in the PKGINFO file, and have a pacman -Qi operation display the raw string as stored by libalpm. Signed-off-by: Dan McGee <dan@archlinux.org>
2007-09-16Remove the DB consistency check from pacman and libalpm.Chantry Xavier
This reverts commit dfc85cb5f516ffbcff557522e9703c5c7d88b047 and b6f3fe6957d0206485eac98fb2120578b75d0058. This DB check is already in testdb (among others). Also testdb now uses the db path set at make time by default, so specifying the db path is optional. Signed-off-by: Chantry Xavier <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2007-08-23libalpm: add newlines to all strings passed to log callbackDan McGee
This allows us to remove the hack in the frontend where we added a newline to everything coming out of the pm_printf functions, and instead let the developer put newlines where they want them. This should be the last hangover of that auto-newline stuff. Signed-off-by: Dan McGee <dan@archlinux.org>
2007-08-16Remove support for SHA1 from pacman.Andrew Fyfe
There's no need for a second hashing algorithm. MD5 serves the purpose of verifying that a package file hasn't been corrupted during download. Signed-off-by: Andrew Fyfe <andrew@neptune-one.net> Signed-off-by: Dan McGee <dan@archlinux.org>
2007-08-14package.h: implement origin_data union in pmpkg_t structDan McGee
We were using a void *data element in pmpkg_t before, which is unsafe by its nature of being untyped. Reimplement data as origin_data being a union that can hold either a path to a package file or a pointer to a cache database, and make the other necesary updates in the code to reflect this. See package.h for details. Signed-off-by: Dan McGee <dan@archlinux.org>