Get continuous frames from camera filter when source filter stops sending?

Question

enter image description here

Hi All,

I have a transform filter which takes two inputs, one from camera and other from a file source. Inside my transform filter I am blending the inputs from two sources.

Transform filter is derived from CTransformFilter

class CWMTransformFilter : public CTransformFilter

and extra pin is derived from: CTransformInputPin(which inturn derives from CBaseInputPin)

class CFileInputPin : public CTransformInputPin

In my case what is happening is, if the file source is small (assume 10 secs), I get input from camera also for 10 secs, later camera stops sending frames to the input pin.

So what i now need is :
1. How to inform camera to send frames even when source filter stops sending ?
2. How to restart the source filter when the playing of the source file is stopped? (something like playing file in loop)

Update:

STDMETHODIMP CFileInputPin::EndOfStream()
{
    //return CTransformInputPin::EndOfStream();
    return S_OK;
}

STDMETHODIMP CFileInputPin::Receive(IMediaSample* pSample)
{
    HRESULT hr;
    BYTE* pBufferIn;
    long lBufferLength, lBufferSize;
    hr = CBaseInputPin::Receive(pSample);
    if (FAILED(hr))
    {
        printf("Error !!\n");
        return hr;
    }
    hr = pSample->GetPointer(&pBufferIn);
    DWORD stat = WaitForSingleObject(m_pFil->m_QSem,0L);
    BOOL bSem = FALSE;
    if( WAIT_OBJECT_0 == stat )
    {
        BYTE *pBuf = (BYTE *) malloc( Wsize*2 );
        memcpy(pBuf,pBufferIn,Wsize); //lBufferLength);
        sEncodedFrame CurFrame={pBuf,Wsize};
        m_pFil->m_Q.push(CurFrame); //push it onto the queue

        bSem = ReleaseSemaphore(m_pFil->m_QSem,1,NULL);
        if(!bSem)
        {
            printf("ReleaseSemaphore error: %d \n", GetLastError());
        }

        return S_OK;
    }
    else
    {
        printf("Cant Receive frame 0x%x \n",stat);
        return E_FAIL;
    }
    return S_OK;
}

HRESULT CWMTransformFilter::Transform(IMediaSample *pSource, IMediaSample *pDest)
{
    unsigned char r,g,b;
    unsigned char y,u,v;

    BYTE *pBufferIn, *pBufferOut, *pBuf;
    HRESULT hr = pSource->GetPointer(&pBufferIn);
    hr = pDest->GetPointer(&pBufferOut);
    if (FAILED(hr))
    {
        return hr;
    }
    long srclen = pSource->GetActualDataLength();
    long dstlen = pDest->GetActualDataLength();

    LONG pLastCnt;
    BOOL bSem = FALSE;
    //printf("Waiting to fill buffer %d\n",pSource);
    //return S_OK;
    //try
    //{
    while(1)
    {
        //if(1)
        DWORD ret = WaitForSingleObject(m_QSem,0L);
        if(ret != WAIT_OBJECT_0)
        {
            printf("Get error %d \n",GetLastError());
        }
        if( WAIT_OBJECT_0 == ret )
        {
            
            sEncodedFrame Frame;
            if( m_Q.empty() == false )
            {
                Frame = m_Q.front();
                m_Q.pop();
                pBuf = (BYTE*) malloc(dstlen*2);
                pLastCnt = Frame.iValidSize;
                printf("Copy onto queue \n");
                memcpy(pBuf,Frame.pFrame,pLastCnt); //Frame.iValidSize);
                free(Frame.pFrame);
                //delete &Frame;
                bSem = ReleaseSemaphore(m_QSem,1,NULL);
                if(!bSem)
                {
                    printf("ReleaseSemaphore error: %d \n", GetLastError());
                }
                hr = S_OK;
                break;
            }
            else
            {
                bSem = ReleaseSemaphore(m_QSem,1,NULL);
                if(!bSem)
                {
                    printf("ReleaseSemaphore error: %d \n", GetLastError());
                }
                //return S_OK;
                hr = E_FAIL;
            }
        }
        else
        {
            //return S_OK;
            hr = E_FAIL;
        }
    }

    for(i = 0; i < windowWidth*2*windowHeight ; i+=4)
    {
     y = pBufferIn[i];
     u = pBufferIn[i+1];
     v = pBufferIn[i+3];
     r = y + 1.4075 * (v - 128);
     g = y - 0.3455 * (u - 128) - (0.7169 * (v - 128));
     b = y + 1.7790 * (u - 128);

    if(((r > b) &&(g > b)) && (g <= 200) )
    {
        pBufferIn[i] = pBuf[i];
        pBufferIn[i+1] = pBuf[i+1];
        pBufferIn[i+2] = pBuf[i+2];
        pBufferIn[i+3] = pBuf[i+3];
    }
}

    // Process the data.
    memcpy(pBufferOut,pBufferIn,pSource->GetSize()); //after blend

    pDest->SetActualDataLength(pDest->GetSize());
    pDest->SetSyncPoint(TRUE);
    return S_OK;
}

CFileInputPin::Receive is where I receive samples from file input pin,
CFileInputPin::EndOfStream(), notifies that samples are completed.
CWMTransformFilter::Transform(), is where the samples are given out to the outpin to renderer.

Thanks,
Shyam

Roman R. Roman R. · Accepted Answer · 2013-07-28T19:09:47

There are many tricky things taken into the question.

First of all, the first point of interest is custom dual input filter. Whatever input filters do, this transform filter can rule things out - it solely depends on its implementation whether it is going to allow both input legs stream, or it will block one of it. The common (typical) rule is that if a filter has 2+ inputs, it is either of the two:

One of the streams is master, and the other inputs are either taken into processing if they make sense or they are discarded
The filter blocks input input legs to keep getting data with matching time stamps, then merges streams over the course of its internal processing.

The input streams are typically sequences of samples followed by EOS notification. In particular, a freeze might take place if one of the sources does not send EOS, or transform filter does not process it properly.

The second big issue is seeking. You normally don't seek a part of the graph. However, here it is exactly what you are trying to do. You can try seek the source filter independently locating its own seeking interfaces, or otherwise you can implement a buffer and hold everything it sends and looping this infinitely once source sends EOS. There is no advice on this - you decide what is appropriate in your scenario.

Another option is to split graphs and bridge them so that you could seek/restart source graph the regular way.

Get continuous frames from camera filter when source filter stops sending?

1 Answers