Commit graph

79 commits

Author SHA1 Message Date
InsanePrawn
73dc5a287a packages: discover_pkgbuilds: use threading as backend for Parallel instead of loky for speed
benchmark: `time kupferbootstrap -vW packages list`

uncached srcinfos:
- threading: ~83s
- multiprocessing: ~86s
- loky: ~144s (~2x)

cached srcinfos:
- threading: ~0.6s
- multiprocessing: ~0.8s
- loky: ~50s (~100x)
2022-11-09 20:19:04 +01:00
InsanePrawn
3b8242be19 packages: parse_pkgbuilds: cache makepkg --printsrcinfo output to SRCINFO, tie to PKGBUILD checksum in srcinfo_meta.json 2022-11-09 20:19:04 +01:00
InsanePrawn
f77aa4f2a2 config: add listings of devices and flavours to config profile init aka prompt_profile() and --no-parse flag
This is slow-ish without SRCINFO caching
2022-11-09 20:19:04 +01:00
InsanePrawn
adcdf38c14 packages: add flavour.py: initial implementation for Flavour scanning 2022-11-09 20:19:04 +01:00
InsanePrawn
6f9a013c2e packages: add cmd_devices_list() 2022-11-09 20:19:04 +01:00
InsanePrawn
75c832cbfb packages: get_devices(): use lazy scanning, scan only device repository 2022-11-09 20:19:04 +01:00
InsanePrawn
ddc92012b8 packages: discover_pkgbuilds(): introduce ability for partial scanning by limiting repositories 2022-11-09 20:19:04 +01:00
InsanePrawn
2a20f6c45a config: add Config.enforce_profile_flavour_set() 2022-11-09 20:19:04 +01:00
InsanePrawn
d3cdd64aea packages: improve ux around cli and filter_pkgbuilds() to indicate that the query was empty or wrong 2022-11-09 20:19:04 +01:00
InsanePrawn
bca1e29648 packages/pkgbuild: add pkgdesc description parsing 2022-11-09 20:19:04 +01:00
InsanePrawn
1fd84bb9c2 packages/pkgbuild: add get_pkgbuild_by_name(name, lazy=True) for easy cache access 2022-11-09 20:19:04 +01:00
InsanePrawn
e4320578ef packages: init_pkgbuilds(): add lazy param and behaviour 2022-11-09 20:19:04 +01:00
InsanePrawn
69404a4267 packages/build: check_package_built(): add init_prebuilts(arch) 2022-11-09 20:19:04 +01:00
InsanePrawn
98140565ef packages: parse_pkgbuild(): handle split packages properly 2022-11-09 20:19:04 +01:00
InsanePrawn
b80f42d48a packages: get_makepkg_env(): don't use host env, but provide standard PATH variable 2022-11-09 20:19:04 +01:00
InsanePrawn
c3e4b5c108 packages/device: get_device: check if device pkgbuild dir exists upfront and raise clear exception if not instead of asserting 2022-11-09 20:11:20 +01:00
InsanePrawn
4f7cb8f516 packages/device: implement Device.parse_deviceinfo(), add deviceinfo tests 2022-11-09 20:11:20 +01:00
InsanePrawn
81f88dd636 packages: clean up check_package_built() and use RemotePackage.acquire() in try_download_package() 2022-11-09 20:11:20 +01:00
InsanePrawn
a2c8868d61 packages/build: add_file_to_repo(): add remove_original=True parameter, clean up add_package_to_repo() 2022-11-09 20:11:19 +01:00
InsanePrawn
57fec8fd91 packages/build: split out init_local_repo() from init_prebuilts(), use in add_file_to_repo() 2022-11-09 20:11:19 +01:00
InsanePrawn
bf420a73be distro: refactor BinaryPackage, Repo and Distro into generics for Local and Remote 2022-11-09 20:11:19 +01:00
InsanePrawn
dbe3dc91a3 distro/package: refactor class PackageInfo into class BinaryPackage(PackageInfo) 2022-11-09 19:39:59 +01:00
InsanePrawn
daa7b68738 packages: split up __init__.py into build.py and cli.py 2022-11-09 19:39:59 +01:00
InsanePrawn
d7f61f6475 packages: move filter_packages() to pkgbuild, rename to filter_pkgbuilds() 2022-11-09 19:39:59 +01:00
InsanePrawn
a3ec35bcd6 config: introduce rust cache 2022-11-09 19:39:59 +01:00
InsanePrawn
47e74fb415 config: introduce per-arch persisted ccache dir 2022-11-09 19:39:59 +01:00
InsanePrawn
d3e2224b02 pkgbuild.discover_pkgbuilds(): warn and skip directories that don't contain a PKGBUILD 2022-11-09 19:39:59 +01:00
InsanePrawn
5cb747aa26 packages and image: wrap more upfront on missing binaries 2022-11-09 19:39:59 +01:00
InsanePrawn
cdc803031e packages.filter_packages(): only filter by arch if arch is not None 2022-11-09 19:39:59 +01:00
InsanePrawn
8fc33c85cf packages.cmd_sideload(): fix escape of --overwrite=* 2022-11-09 19:39:59 +01:00
InsanePrawn
32f5fe643f packages: respect package arches before and during building 2022-11-09 19:39:59 +01:00
InsanePrawn
a778f0786e packages.check_package_version_built(): use Pkgbuild.get_filename() instead of running makepkg --packagelist 2022-11-09 19:39:59 +01:00
InsanePrawn
f2ccf06c5b pkgbuild: add get_filename(arch) 2022-11-09 19:39:59 +01:00
InsanePrawn
fb4a12c464 packages.filter_packages(): optionally check package arch 2022-11-09 19:39:59 +01:00
InsanePrawn
bb14c4d779 packages: use user 'kupfer' in chroots for building 2022-11-09 19:39:59 +01:00
InsanePrawn
8ad18c00a6 global: refactor to use config.{file,runtime}.$member instead of config.file["$member"] 2022-11-09 19:39:59 +01:00
InsanePrawn
3d6aa98c21 packages: circumvent git dubious ownership errors in pkgbuilds.git due to chrootery 2022-11-02 00:38:18 +01:00
InsanePrawn
d03cb39358 Pkgbuild: add refresh_sources() 2022-11-02 00:38:18 +01:00
InsanePrawn
558993b89e packages/pkgbuild: cache parsed pkgbuilds by path, add get_pkgbuild_by_path(), Pkgbuild.update(pkgb) 2022-11-02 00:38:18 +01:00
InsanePrawn
2db8a0a0cb packages: cleanup unnecessary bits 2022-08-25 13:52:35 +02:00
InsanePrawn
5a794ba3dd packages: get_makepkg_env(): actually return env... 2022-08-23 17:38:58 +02:00
InsanePrawn
39be2b2fb6 packages/test_device: add test for initialise_pkgbuilds_dir fixture 2022-08-20 04:51:25 +02:00
InsanePrawn
f5e3fa46ad packages/ and image.py: less wrapping, remove hardcoded 'aarch64' where possible 2022-08-20 04:32:33 +02:00
InsanePrawn
8a7f78261f packages: add device.py and initial tests 2022-08-20 04:32:33 +02:00
InsanePrawn
1c6689f710 packages.cmd_sideload(): fix string escape '\*' -> '\\*' 2022-08-19 17:18:18 +02:00
InsanePrawn
e001d107c2 move packages.{clone_pkbuilds(),discover_packages(),init_pkgbuilds()} to packages.pkgbuild
also rename discover_packages() to discover_pkgbuilds()
2022-08-19 17:13:03 +02:00
InsanePrawn
5baaaaa180 packages.discover_packages(): clean up caching 2022-08-19 16:55:53 +02:00
InsanePrawn
7d9f1b9ed8 pkgbuild.parse_pkgbuild() make config parameter optional 2022-08-19 16:55:17 +02:00
InsanePrawn
859b08df6a packages.build_enable_qemu(): fixup for usage without wrapper, drop buildchroot requirement 2022-08-16 03:38:33 +02:00
InsanePrawn
dbf65b44df parse_pkgbuild(): update basepkg version before copying to subpkgs 2022-08-16 02:35:40 +02:00