Angular reactive form validity with dynamic form group controls

Question

I have an angular reactive form in which I am listening to valueChanges for only a valid form state. However, I also need to listen to valueChanges on my form control and add/remove controls depending on the value. The problem is that when I change the value of control my formGroup subscription is triggering because technically the form is still valid until the controls have been added. How do I make the formGroup subscription wait until the controls have been added?

    this.formGroup
      .get('control')?.valueChanges
      .subscribe(value => {
        this.removeControls();
        this.addControls(value);
      });

    this.formGroup.valueChanges.pipe(
      filter(_ => this.formGroup.valid)
    ).subscribe(values => this.update(values));