Executing tasks in series which creates file dynamically in Gulp 4

Question

I've recently migrated from using Gulp 3 to Gulp 4. Introducing the new tasks like gulp.src() and gulp.parallel() adds more flexibility and ease of use.
Here is my code attempting to execute tasks in series.

1. Asynchronous task which creates files dynamically

   let css_purge_libs = async () => {
   var pacs = ["!bootstrap", "bulma", "animate.css"];
   var paths = await fetch_resources(pacs, "bower", "css|sass");
   return gulp.src(paths)
     .pipe(before(true, config.css.destination.temp + "/lib")())
     .pipe(P.if(/\.sass$/gm, P.sass().on('error', P.sass.logError)))
     .pipe(P.purgecss(config.css.purgecss))
     .pipe(gulp.dest(config.css.destination.temp + "/lib"));
   };
   

wrote above code in order to fetch .css or .sass files from bower specific packages which there was no mentioned of main property in bower.json(don't need an additional plugin like wiredep). files will be created on the fly in temp folder.

2. Building, concat and sourcemap generation

   let css_build = () => {
   let paths = [`${config.css.destination.temp}/**/*.css`, `${config.css.destination.dev}**/*.css`];
   return gulp.src(paths)
     .pipe(before(true, config.css.destination.build)())
     .pipe(P.postcss(config.css.postcss.plugins))
     .pipe(P.if(prod, P.sourcemaps.init()))
     .pipe(P.if(prod, P.cssnano()))
     .pipe(P.if(prod, P.concat("ariyana-build.css")))
     .pipe(P.if(prod, P.rev()))
     .pipe(P.if(prod, P.sourcemaps.write(".")))
     .pipe(gulp.dest(config.css.destination.build));
   }
   

fetched resources and project resources will be sent through a stream for processing.

3. Gulp task definitions

   gulp.task('css:purge-libs', css_purge_libs); 
   gulp.task('css:build', css_build);
   gulp.task("serve:dev", gulp.series(css_purge_libs, css_build, serve_dev));
   

   Problem Scenario
   

When i issue gulp serve:dev, what happens is attached in below logs.

[00:19:19] Using gulpfile D:\Enterprise\test\gulpfile.js
[00:19:19] Starting 'serve:dev'...
[00:19:19] Starting 'css_purge_libs'...
[00:19:21] Finished 'css_purge_libs' after 1.86 s
[00:19:21] Starting 'css_build'...
[00:19:25] bower_components\bulma\bulma.sass
[00:19:25] app\styles\bootstrap-reboot.css
[00:19:25] bower_components\animate.css\animate.css
[00:19:26] app\styles\main.css
[00:19:26] Finished 'css_build' after 4.69 s
[00:19:26] Starting 'serve_dev'...
[00:19:26] Finished 'serve_dev' after 1.57 ms
[00:19:26] Finished 'serve:dev' after 6.57 s

looks like gulp did not picked up the dynamically created files in the .tmp/styles/lib css resources. I could think a reason what happens because the IO op happens under the hood could be an async ops so gulp does not have a clue whether the op is done

Answer scenario

let sloop = ()=> {
  return new Promise(resolve=> {
    setTimeout(resolve, 2000);
  });
}

when i add a little bit of a tweek which brings a delay and introduced in the task definition

gulp.task('css:purge-libs', css_purge_libs); 
gulp.task('css:build', css_build);
gulp.task("serve:dev", gulp.series(css_purge_libs, sloop, css_build, serve_dev));

this came as the output log which i was expected.

[00:28:58] Using gulpfile D:\Enterprise\test\gulpfile.js
[00:28:58] Starting 'serve:dev'...
[00:28:58] Starting 'css_purge_libs'...
[00:29:00] Finished 'css_purge_libs' after 1.8 s
[00:29:00] Starting 'sloop'...
[00:29:00] bower_components\bulma\bulma.sass
[00:29:00] bower_components\animate.css\animate.css
[00:29:02] Finished 'sloop' after 2.18 s
[00:29:02] Starting 'css_build'...
[00:29:08] .tmp\styles\lib\animate.css
[00:29:08] .tmp\styles\lib\bulma.css
[00:29:08] app\styles\bootstrap-reboot.css
[00:29:08] app\styles\main.css
[00:29:08] Finished 'css_build' after 6.43 s
[00:29:08] Starting 'serve_dev'...
[00:29:08] Finished 'serve_dev' after 2.96 ms
[00:29:08] Finished 'serve:dev' after 10 s

introducing a new delay between async ops solved my problem

Questions

1. Why is this happening ?(please give me in depth explanation because i'm searching for the internals)
2. Why gulp stream can't poll if the file was done creating or not.
3. Is there any better way to handle these kinds of scenario ?

Answer 1

Unfortunately, this is a known issue in gulp that doesn't have a workaround: https://github.com/gulpjs/gulp/issues/1421

What you return from your task tells gulp how to wait: you can return a Stream (gulp waits for it to finish), or a Promise (gulp waits for it to resolve), etc. What you cannot do, as you are trying to do, is return a Promise for a Stream.

The solution, for now, is to manually wait for your gulp stream to end inside your async function, like so:

let myTask = async () => {
    let files = await getFilePaths();
    return new Promise((resolve, reject) => {
        return gulp.src(files)
            .pipe(gulp.dest('someplace'))
            .on('end', resolve)
            .on('error', reject);
    });
};

It's ugly, but there's not currently a better way to do both standard async calls and stream management in a single task.