Yes, it's possible to make changes without redeploying the app.
Some details:
Check Run the package document and we can find:
1.The zip package won't be extracted to D:\home\site\wwwroot, instead it will be uploaded directly to D:\home\data\SitePackages.
2.A packagename.txt which contains the name of the ZIP package to load at runtime will be created in the same directory.
3.App Service mounts the uploaded package as the read-only wwwroot directory and runs the app directly from that mounted directory. (That's why we can't edit the read-only wwwroot directory directly)
So my workaround is:
1.Navigate to D:\home\data\SitePackages in via kudu- debug console:

Download the zip(In my case it's 20200929072235.zip) which represents your deployed app, extract this zip file and do some changes to web.config file.

2.Zip those files(choose those files and right-click...) into a childtest.zip, please follow my steps carefully here!!! The folder structure of Run-from-package is a bit strange!!!

3.Then zip the childtest.zip into parenttest.zip(When uploading the xx.zip, the kudu always automatically extra them. So we have to zip the childtest.zip into parenttest.zip first)

4.Drag and drop local parenttest.zip into online SitePackages folder in kudu-debug console and we can get a childtest.zip now:

5.Modify the packagename.txt, change the content from 20200929072235.zip to childtest.zip and Save:

Done~
Check and test:
Now let's open App Service Editor to check the changes:

In addition: Though it answers the original question, I recommend using other deployment methods(web deploy...) as a workaround. It could be much easier~
run from package? It's possible to do what you want, but the workaround is a bit complex. So I think a redeployment would be better if you don't have specific reason to avoid redeploying... (PS: I'll post my detailed workaround later for you) - LoLance