How to detect changes in local files in bitbake?

Question

Let's say I have a simple recipe that only adds one script file:

LICENSE = "CLOSED"
SRC_URI += "file://test.sh"
S = "${WORKDIR}"

do_install () {
    mkdir -p ${D}/usr/bin
    cp -r test.sh ${D}/usr/bin
}

If I modify test.sh and re-run bitbake, it does nothing because it doesn't realize anything has changed. Is there any way to get the recipe to check if there are changes in SRC_URI files? And recompile if changes are found. SRC_URI[md5sum] is not supported for local files.

EDIT: My original question was a bit flawed, due to my somewhat unrealistic test setup. But it did show an interesting feature in bitbake.

I switched between two versions of test.sh, where I just added or removed one character. What happened was this:

Run bitbake -> test-recipe is compiled
Edit test.sh by adding one char
Run bitbake -> test-recipe is compiled
Edit test.sh by removing the char
Run bitbake -> test-recipe is not compiled
Edit test.sh by adding the same char as before
Run bitbake -> test-recipe is not compiled

So if test.sh changes to a previous version, bitbake does not notice the change. There must be some sort of weird caching going on somewhere. What this means is that the recipe I used originally worked, but only if the changes to test.sh are new, actual changes and not the artificial ones I used to test the recipe.

Can you store test.sh in a git repository? If so, then it is done for you as the git checkout hash will cause rebuilds when changed. Also, you could build by doing the following... bitbake test-package bitbake image-name — Peter Buelow
I am aware of that option, but unfortunately in this case using a git repository is not an option. Also, just running "bitbake test-package" will not trigger a rebuild. I have to run "bitbake -c cleansstate test-package" first. But I would prefer just to run "bitbake image-name" and have all recipes with updated files be recompiled automatically. That's what I was looking for with the original question. — Sampo

User3219 User3219 · Accepted Answer · 2018-09-27T09:56:57

As already stated, it is better suited to have the recipe under version control so that new commits will cause a state change and the SRCREV would hence be altered and trigger a new fetch/compile/install

I just tried your recipe and added it to my core-image-minimal image and I did not see any problems with detecting changes in the test.sh script. This is my slightly modified recipe I am using.

LICENSE = "CLOSED"
SRC_URI += "file://test.sh"
S = "${WORKDIR}"

PR = "0"

do_install () {
    install -d ${D}/usr/bin
    install -m 0644 test.sh ${D}/usr/bin
}

Save it as test.bb inside an existing layer. Add this recipe to your image recipe:

IMAGE_INSTALL += "test"

Then bitbake your image. I used the minimal base image.

bitbake core-image-minimal.

Edit test.sh and re-run the previous bitbake. You should see bitbake install your new script into your image located at tmp/deploy/images/<your_target>/core-iamge-minimal.tar.xz

If that fails, then you can always manually bump the PR value in your recipe.

After seeing your edit:

So if test.sh changes to a previous version, bitbake does not notice the change. There must be some sort of weird caching going on somewhere.

bitbake caches all inputs so that different iterations of your script can be quickly pulled from the shared-state cache. You should see a folder inside your tmp folder labeled sstate-cache. This is where your previously compiled script resides. In short - this is expected behavior and everything is fine.

How to detect changes in local files in bitbake?

1 Answers