Geek Answers Handbook

Change motion vectors in ffmpeg H.264 decoder

For research purposes, I am trying to change the H.264 (MV) motion vectors for each P- and B-frame before motion compensation during the decoding process. For this I use FFmpeg. An example of a modification is replacing each MV with its original spatial neighbors, and then using the resulting MVs to compensate for the motion, not the original ones. Please guide me accordingly.

So far, I have managed to perform a simple modification of MV in the file /libavcodec/h264_cavlc.c . In the function ff_h264_decode_mb_cavlc () , changing the variables mx and my , for example, by increasing their values ​​modifies the MVs used during decoding.

For example, as shown below, the values ​​of mx and my are increased by 50, thus lengthening the MVs used in the decoder.

mx += get_se_golomb(&s->gb)+50;
my += get_se_golomb(&s->gb)+50;

However, in this regard, I do not know how to access the neighbors mx and my for my spatial average analysis, which I mentioned in the first paragraph. I believe the key to this is manipulating the array, mv_cache .

Another experiment that I performed was in the file libavcodec / error_resilience.c . Based on the guess_mv () function, I created a new mean_mv () function that runs in ff_er_frame_end () during the first if-expression. This first if-statement completes the ff_er_frame_end () function if one of the conditions is zero errors (s-> error_count == 0). However, at this point I decided to insert my mean_mv () function, , . , , , . , , - .

if. - , mean_mv (s).

if(!s->error_recognition || s->error_count==0 || s->avctx->lowres ||
       s->avctx->hwaccel ||
       s->avctx->codec->capabilities&CODEC_CAP_HWACCEL_VDPAU ||
       s->picture_structure != PICT_FRAME || // we dont support ER of field pictures yet, though it should not crash if enabled
       s->error_count==3*s->mb_width*(s->avctx->skip_top + s->avctx->skip_bottom)) {
        //av_log(s->avctx, AV_LOG_DEBUG, "ff_er_frame_end in er.c\n"); //KG
        if(s->pict_type==AV_PICTURE_TYPE_P)
            mean_mv(s);
        return;

mean_mv(), guess_mv().

static void mean_mv(MpegEncContext *s){
    //uint8_t fixed[s->mb_stride * s->mb_height];
    //const int mb_stride = s->mb_stride;
    const int mb_width = s->mb_width;
    const int mb_height= s->mb_height;
    int mb_x, mb_y, mot_step, mot_stride;

    //av_log(s->avctx, AV_LOG_DEBUG, "mean_mv\n"); //KG

    set_mv_strides(s, &mot_step, &mot_stride);

    for(mb_y=0; mb_y<s->mb_height; mb_y++){
        for(mb_x=0; mb_x<s->mb_width; mb_x++){
            const int mb_xy= mb_x + mb_y*s->mb_stride;
            const int mot_index= (mb_x + mb_y*mot_stride) * mot_step;
            int mv_predictor[4][2]={{0}};
            int ref[4]={0};
            int pred_count=0;
            int m, n;

            if(IS_INTRA(s->current_picture.f.mb_type[mb_xy])) continue;
            //if(!(s->error_status_table[mb_xy]&MV_ERROR)){
            //if (1){
            if(mb_x>0){
                mv_predictor[pred_count][0]= s->current_picture.f.motion_val[0][mot_index - mot_step][0];
                mv_predictor[pred_count][1]= s->current_picture.f.motion_val[0][mot_index - mot_step][1];
                ref         [pred_count]   = s->current_picture.f.ref_index[0][4*(mb_xy-1)];
                pred_count++;
            }

            if(mb_x+1<mb_width){
                mv_predictor[pred_count][0]= s->current_picture.f.motion_val[0][mot_index + mot_step][0];
                mv_predictor[pred_count][1]= s->current_picture.f.motion_val[0][mot_index + mot_step][1];
                ref         [pred_count]   = s->current_picture.f.ref_index[0][4*(mb_xy+1)];
                pred_count++;
            }

            if(mb_y>0){
                mv_predictor[pred_count][0]= s->current_picture.f.motion_val[0][mot_index - mot_stride*mot_step][0];
                mv_predictor[pred_count][1]= s->current_picture.f.motion_val[0][mot_index - mot_stride*mot_step][1];
                ref         [pred_count]   = s->current_picture.f.ref_index[0][4*(mb_xy-s->mb_stride)];
                pred_count++;
            }

            if(mb_y+1<mb_height){
                mv_predictor[pred_count][0]= s->current_picture.f.motion_val[0][mot_index + mot_stride*mot_step][0];
                mv_predictor[pred_count][1]= s->current_picture.f.motion_val[0][mot_index + mot_stride*mot_step][1];
                ref         [pred_count]   = s->current_picture.f.ref_index[0][4*(mb_xy+s->mb_stride)];
                pred_count++;
            }

            if(pred_count==0) continue;

            if(pred_count>=1){
                int sum_x=0, sum_y=0, sum_r=0;
                int k;

                for(k=0; k<pred_count; k++){
                    sum_x+= mv_predictor[k][0]; // Sum all the MVx from MVs avail. for EC
                    sum_y+= mv_predictor[k][1]; // Sum all the MVy from MVs avail. for EC
                    sum_r+= ref[k];
                    // if(k && ref[k] != ref[k-1])
                    // goto skip_mean_and_median;
                }

                mv_predictor[pred_count][0] = sum_x/k;
                mv_predictor[pred_count][1] = sum_y/k;
                ref         [pred_count]    = sum_r/k;
            }

            s->mv[0][0][0] = mv_predictor[pred_count][0];
            s->mv[0][0][1] = mv_predictor[pred_count][1];

            for(m=0; m<mot_step; m++){
                for(n=0; n<mot_step; n++){
                    s->current_picture.f.motion_val[0][mot_index + m + n * mot_stride][0] = s->mv[0][0][0];
                    s->current_picture.f.motion_val[0][mot_index + m + n * mot_stride][1] = s->mv[0][0][1];
                }
            }

            decode_mb(s, ref[pred_count]);

            //}
        }
    }
}

, .

+5
1

FFMPEG.

, FFMPEG ( , ), .

№ 1
, - FFMPEG (a.k.a s), . , .

№ 2
, ( , )

mx 50

, 50 - . , , F- . +/- 8 ( +/- 16), , , +50 , .

mean_mv() , . .

+2

More articles:


All Articles