�ɲɾ�����ӯ�����һ��ˣ��������С���˴��ͣ�������P���ҹ��ñ˽��ά�Բ��������˸߸ԣ�������ơ��ҹ��ñ�����ά�Բ���ˡ���˳^�ӣ������ӡ�
���ͯj�ӣ��ƺ���ӣ�
? PNG ?%k25u25%fgd5n!? PNG ?%k25u25%fgd5n!? PNG ?%k25u25%fgd5n!? PNG ?%k25u25%fgd5n!PK #2]1? ? Appendix-Alt-Refs.mdnu [ # ALTREF Pictures - Temporal filtering Appendix
## 1. Description of the algorithm
ALTREFs are non-displayable pictures that are used as reference for
other pictures. They are usually constructed using several source frames
but can hold any type of information useful for compression and the
given use-case. In the current version of SVT-AV1, temporal filtering of
adjacent video frames is used to construct some of the ALTREF pictures.
The resulting temporally filtered pictures will be encoded in place of
or in addition to the original sources. This methodology is especially
useful for source pictures that contain a high level of noise since the
temporal filtering process will produce reference pictures with reduced
noise level.
Temporal filtering is currently applied to the base layer picture of
each mini-GOP (e.g. source frame position 16 in a mini-GOP in a 5-layer
hierarchical prediction structure). In addition, filtering of the
key-frames and intra-only frames is also supported.
Two important parameters control the temporal filtering operation:
```altref_nframes``` which denotes the number of pictures to use for
filtering, also referred to as the temporal window, and
```altref_strength``` which denotes the strength of the filter.
The diagram in Fig. 1 illustrates the use of 5 adjacent pictures
(```altref_nframes = 5```), 2 past, 2 future and one central pictures, in
order to produce a single filtered picture. Motion estimation is applied
between the central picture and each future or past pictures generating
multiple motion-compensated predictions. These are then combined using
adaptive weighting (filtering) to produce the final noise-reduced
picture.

##### Fig. 1. Example of motion estimation for temporal filtering in a temporal window consisting of 5 adjacent pictures
Since a number of adjacent frames are necessary (identified by the
parameter *altref_nframes*) the Look Ahead Distance (LAD) needs to be
adjusted according to the following relationship:

For instance, if the ```miniGOPsize``` is set to 16 pictures, and
```altref_nframes``` is 7, a ```LAD``` of 19 frames would be required.
When applying temporal filtering to ALTREF pictures, an Overlay picture
is usually necessary. This picture corresponds to the same original
source picture but can use the temporally filtered version of the source
picture as a reference.
### Steps of the temporal filtering algorithm:
#### Step 1: Building the list of source pictures
As mentioned previously, the temporal filtering algorithm uses multiple
frames to generate a temporally denoised or filtered picture at the
central picture location. If enough pictures are available in the list
of source picture buffers, the number of pictures used will generally be
given by the ```altref_nframes``` parameter, unless not enough frames are
available (e.g. end of sequence). This will correspond to ```floor(altref_ nframes/2)``` past pictures and ```floor((altref_ nframes - 1)/2)``` future pictures in addition to
the central picture. Therefore, if the ```altref_nframes``` is an even
number, the number of past pictures will be larger than the number of
future pictures. Therefore, non-symmetric temporal windows are allowed.
However, in order to account for illumination changes, which might
compromise the quality of the temporally filtered picture, an adjustment
of the ```altref_nframes``` is conducted to remove cases where a
significant illumination change is found in the defined temporal window.
This algorithm first computes and accumulates the absolute difference
between the luminance histograms of adjacent pictures in the temporal
window, starting from the first past picture to the last past picture
and from the first future picture to the last future picture. Then,
depending on a threshold, ```ahd_thres```, if the cumulative difference
is high enough, edge pictures will be removed. The current threshold is
chosen based on the picture width and height:

After this step, the list of pictures to use for the temporal filtering
is ready. However, given that the number of past and future frames can
be different, the index of the central picture needs to be known.
#### Step 2: Source picture noise estimation and strength adjustment
In order to adjust the filtering strength according to the content
characteristics, the amount of noise is estimated from the central
source picture. The algorithm considered is based on a simplification of
the algorithm proposed in [\[1\]](#ref-1). The standard deviation (sigma) of the
noise is estimated using the Laplacian operator. Pixels that belong to
an edge (i.e. as determined by how the magnitude of the Sobel gradients
compare to a predetermined threshold), are not considered in the
computation. The current noise estimation considers only the luma
component.
The filter strength is then adjusted from the input value,
```altref_strength```, according to the estimated noise level, ```noise_level```.
If the noise level is low, the filter strength is decreased. The final
strength is adjusted based on the following conditions:

#### Step 3: Block-based processing
The central picture is split into 64x64 pixel non-overlapping blocks.
For each block, ``` altref_nframes - 1``` motion-compensated
predictions will be determined from the adjacent frames and weighted in
order to generate a final filtered block. All blocks are then combined
to build the final filtered picture.
#### Step 4: Block-based motion estimation and compensation
For each block and each adjacent picture, hierarchical block-based
motion estimation (unilateral prediction) is performed. A similar
version of the open-loop Hierarchical Motion Estimation (HME), performed
in subsequent steps in the encoding process, is applied. The ME motion
estimation produces ¼-pel precision motion-vectors on blocks from 64x64
to 8x8 pixels. After obtaining the motion information, sub-blocks of
size 16x16 are compensated using the AV1 normative interpolation.
Finally, during this step, a small refinement search using 1/8-pel
precision motion vectors is conducted on a 3x3 search window. Motion is
estimated on the luma channel only, but the motion compensation is
applied to all channels.
#### Step 5: Determination of block-based weights
After motion compensation, distortion between the original ()
and predicted (()) sub-blocks of size 16x16 is computed
using the non-normalized variance () of the residual
(), which is computed as follows:
=B_{p}(i,j)-B_{s}(i,j))
}{H*W})
-\mu)^2)
Based on this distortion, sub-block weights, ```blk_fw```, from 0 to 2 are
determined using two thresholds, ```thres_low``` and ```thres_high```:

Where ```thres_low = 10000``` and ```thres_high = 20000```.
For the central picture, the weights are always 2 for all blocks.
#### Step 6: Determination of pixel-based weights
After obtaining the sub-block weights, a further refinement of the
weights is computed for each pixel of the predicted block. This is based
on a non-local means approach.
First, the Squared Errors, ), between the predicted
and the central block are computed per pixel for the Y, U and V
channels. Then, for each pixel, when computing the Y pixel weight, a
neighboring sum of squared errors, , corresponding
to the sum of the Y squared errors on a 3x3 neighborhood around the
current pixel plus the U and V squared errors of the current pixel is
computed:

The mean of the ,  is then
used to computed the pixel weight  of the current
pixel location (i,j), which is an integer between {0,16}, and is
determined using the following equation:

Where strength is the adjusted *altref\_strength* parameter. The same
approach is applied to the U and V weights, but in this case, the number
of (se) values added from the Y channel depends on the chroma
subsampling used (e.g. 4 for 4:2:0).
As can be observed from the equation above, for the same amount of
distortion, the higher the strength, the higher the pixel weights, which
leads to stronger filtering.
The final filter weight of each pixel is then given by the
multiplication of the respective block-based weight and the pixel
weight. The maximum value of the filter weight is 32 (2\*16) and the
minimum is 0.
In case the picture being processed is the central picture, all filter
weights correspond to the maximum value, 32.
#### Step 7: Temporal filtering of the co-located motion compensated blocks
After multiplying each pixel of the co-located 64x64 blocks by the
respective weight, the blocks are then added and normalized to produce
the final output filtered block. These are then combined with the rest
of the blocks in the frame to produce the final temporally filtered
picture.
The process of generating one filtered block is illustrated in diagram
of Fig. 2. In this example, only 3 pictures are used for the temporal
filtering ```altref_nframes = 3```. Moreover, the values of the filter
weights are for illustration purposes only and are in the range {0,32}.

##### Fig. 2. Example of the process of generating the filtered block from the predicted blocks of adjacent picture and their corresponding pixel weights.
## 2. Implementation of the algorithm
**Inputs**: list of picture buffer pointers to use for filtering,
location of central picture, initial filtering strength
**Outputs**: the resulting temporally filtered picture, which replaces
the location of the central pictures in the source buffer. The original
source picture is stored in an additional buffer.
**Control macros/flags**:
| **Flag** | **Level (sequence/Picture)** | **Description** |
| ---------------- | ------------- | ------------ |
| tf\_level | Sequence | High-level flag to enable/disable temporally filtered pictures (default: enabled) |
| enable\_overlays | Sequence | Enable overlay frames (default: on) |
### Implementation details
The current implementation supports 8-bit and 10-bit sources as well as
420, 422 and 444 chroma sub-sampling. Moreover, in addition to the C
versions, SIMD implementations of some of the more computationally
demanding functions are also available.
Most of the variables and structures used by the temporal filtering
process are located at the picture level, in the PictureControlSet (PCS)
structure. For example, the list of pictures is stored in the
```temp_filt_pcs_list``` pointer array.
For purposes of quality metrics computation, the original source picture
is stored in ```save_enhanced_picture_ptr``` and
```save_enhanced_picture_bit_inc_ptr``` (for high bit-depth content)
located in the PCS.
The current implementation disables temporal filtering on key-frames if
the source has been classified as screen content (```sc_content_detected```
in the PCS is 1).
Due to the fact that HME is open-loop, which means it operates on the
source pictures, HME can only use the source picture which is going to
be filtered after the filtering process has been finalized. The strategy
for synchronizing the processing of the pictures for this case is
similar to the one employed for the determination of the prediction
structure in the Picture Decision Process. The idea is to write to a
queue, the ```picture_decision_results_input_fifo_ptr```, which is
consumed by the HME process.
### Memory allocation
Three uint8_t or uint16_t buffers of size 64x64x3 are allocated: the
accumulator, predictor and counter. In addition, an extra picture buffer
(or two in case of high bit-depth content) is allocated to store the
original source. Finally, a temporary buffer is allocated for high-bit
depth sources, due to the way high bit-depth sources are stored in the
encoder implementation (see sub-section on high bit-depth
considerations).
### High bit-depth considerations
For some of the operations, different but equivalent functions are
implemented for 8-bit and 10-bit sources. For 8-bit sources, uint8_t
pointers are used, while for 10-bit sources, uint16_t pointers are
used. In addition, the current implementation stores the high bit-depth
sources in two separate uint8_t buffers in the EbPictureBufferDesc
structure, for example, ```buffer_y``` for the luma 8 MSB and
```buffer_bit_inc_y``` for the luma LSB per pixel (2 in case of 10-bit).
Therefore, prior to applying the temporal filtering, in case of 10-bit
sources, a packing operation converts the two 8-bit buffers into a
single 16-bit buffer. Then, after the filtered picture is obtained, the
reverse unpacking operation is performed.
### Multi-threading
The filtering algorithm operates independently in units of 64x64 blocks
and is currently multi-threaded. The number of threads used is
controlled by the variable ```tf_segment_column_count```, which depending
the resolution of the source pictures, will allocate more or less
threads for this task. Each thread will process a certain number of
blocks.
Most of the filtering steps are multi-threaded, except the
pre-processing steps: packing (in case of high bit-depth sources) and
unpacking, estimation of noise, adjustment of strength, padding and
copying of the original source buffers. These steps are protected by a
mutex, ```temp_filt_mutex```, and a binary flag, ```temp_filt_prep_done``` in
the PCS structure.
### Relevant files and functions in the codebase
The main source files that implement the temporal filtering operations
are located in Source/Lib/Encoder/Codec, and correspond to:
- EbTemporalFiltering.c
- EbTemporalFiltering.h (header file)
In addition, the logic to build the list of source pictures for the
temporal filtering is located in Source/Lib/Encoder/Codec:
- EbPictureDecisionProcess.c
The table below presents the list of functions implemented in
EbTemporalFiltering.c, grouped by tasks.

## 3. Optimization of the algorithm
The current algorithm provides a good trade-off between compression
efficiency and complexity, and therefore is enabled by default for all
encoding presets, enc-modes, from 0 to 8. No optimizations for higher
speed presets are performed.
## 4. Signaling
If the temporally filtered picture location is of type ```ALTREF_FRAME``` or
```ALTREF2_FRAME```, the frame should not be displayed with the
```show_existing_frame``` strategy and should contain an associated Overlay
picture. In addition, the frame has the following field values in the
frame header OBU:
- ```show_frame = 0```
- ```showable_frame = 0```
- ```order_hint``` = the index that corresponds to the central picture of the ALTREF frame
In contrast, the temporally filtered key-frame will have ```showable_frame```
= 1 and no Overlay picture.
## Notes
The feature settings that are described in this document were compiled at v0.8.3 of the code and may not reflect the current status of the code. The description in this document represents an example showing how features would interact with the SVT architecture. For the most up-to-date settings, it's recommended to review the section of the code implementing this feature.
## References
\[1\] Tai, Shen-Chuan, and Shih-Ming Yang. "A fast method for image
noise estimation using Laplacian operator and adaptive edge detection."
In *2008 3rd International Symposium on Communications, Control and
Signal Processing*, pp. 1077-1081. IEEE, 2008.
PK #2](Pr Pr svt-av1_encoder_user_guide.mdnu [ # Scalable Video Technology for AV1 Encoder (SVT-AV1 Encoder) User Guide
## Table of Contents
1. [Introduction](#introduction)
2. [Sample Application Guide](#sample-application-guide)
- [Input Video Format](#input-video-format)
- [Compressed 10-bit format](#compressed-10-bit-format)
- [Running the encoder](#running-the-encoder)
- [Sample command lines](#sample-command-lines)
- [List of all configuration parameters](#list-of-all-configuration-parameters)
## Introduction
This document describes how to use the Scalable Video Technology for AV1 Encoder (SVT-AV1). In particular, this user guide describes how to run the sample application with the respective dynamically linked library.
## Sample Application Guide
This section describes how to run the sample encoder application that uses the SVT-AV1 Encoder library. It describes the input video format, the command line input parameters and the resulting outputs.
### Input Video Format
The SVT-AV1 Encoder supports the following input formats:
_8-bit yuv420p_\

_10-bit yuv420p10le_\

### Compressed 10-bit format
In order to reduce the size of the input original YUV file, the SVT-AV1 Encoder uses a compressed 10-bit format allowing the software to achieve a higher speed and channel density levels. The conversion between the 10-bit yuv420p10le and the compressed 10-bit format is a lossless operation and is performed using the following steps.
#### Unpack the 10-bit picture
This step consists of separating the 10 bit video samples into 8 bit and 2 bit planes so that each 10-bit picture will be represented as two separate pictures as shown in the figure below. As a result of the operation, the 2 least significant bits of the 10 bits will be written into a full byte.
_10-bit yuv420p10le unpacked_\

#### Compress the 2 bit Plane
The unpacking steps separates the 10bits into a group of 8 bits and a group of 2 bits, where the 2 bits are stored in a byte. In this step, every group of consecutive 4 bytes, each containing 2bits from the unpacking step, are compressed into one byte. As a result, each 10bit picture will be represented as two separate pictures as shown in the figure below.
_10-bit yuv420p10le compressed_\

#### Unroll the 64x64
Now for a faster read of the samples, every 64x64 block of the 2 bit picture should be written into a one dimensional array. Therefore, the top left 64x64 sample block which is now written into a 16 bytes x 64 bytes after the compression of the 2bit samples, will be written into a 1024 bytes x 1 byte array as shown in the picture below.
_64x64 block after 2 bit compression_\

_64x64 block after unrolling_\

### Running the encoder
This section describes how to run the sample encoder application `SvtAv1EncApp.exe` (on Windows\*) or `SvtAv1EncApp` (on Linux\*) from the command line, including descriptions of the most commonly used input parameters and outputs.
The sample application typically takes the following command line parameters:
`-c filename` **[Optional]**
A text file that contains encoder parameters such as input file name, quantization parameter etc. Refer to the comments in the Config/Sample.cfg for specific details. The list of encoder parameters are also listed below. Note that command line parameters take precedence over the parameters included in the configuration file when there is a conflict.
`-i filename` **[Required]**
A YUV file (e.g. 8 bit 4:2:0 planar) containing the video sequence that will be encoded. The dimensions of each image are specified by `-w` and `-h` as indicated below.
`-b filename` **[Optional]**
The resulting encoded bit stream file in binary format. If none specified, no output bit stream will be produced by the encoder.
`-w integer` **[Required]**
The width of each input image in units of picture luma pixels, e.g. 1920
`-h integer` **[Required]**]
The height of each input image in units of picture luma pixels, e.g. 1080
`-n integer` **[Optional]**
The number of frames of the sequence to encode. e.g. 100. If the input frame count is larger than the number of frames in the input video, the encoder will loopback to the first frame when it is done.
`--keyint integer` **[Optional]**
The intra period defines the interval of frames after which you insert an Intra refresh. It is strongly recommended to use (multiple of 8) -1 the closest to 1 second (e.g. 55, 47, 31, 23 should be used for 60, 50, 30, (24 or 25) respectively). When using closed gop (-irefresh-type 2) add 1 to the value above (e.g. 56 instead of 55).
`--rc integer` **[Optional]**
This token sets the bitrate control encoding mode [1: Variable Bitrate, 0: Constant QP OR Constant Rate Factor]. When `--rc` is set to 1.
With `--rc` set to 0, if `--crf` is used then enable-tpl-la is forced to 1, however, if `-q`/`--qp` is used then the encoder will work in CRF mode if `--enable-tpl-la` is set to 1 and in CQP mode (fixed qp offsets regardless of the content) when `--enable-tpl-la` is set to 0.
If a qp/crf value is not specified, a default value is assigned (50).
For example, the following command encodes 100 frames of the YUV video sequence into the bin bit stream file. The picture is 1920 luma pixels wide and 1080 pixels high using the `Sample.cfg` configuration. The QP equals 30 and the md5 checksum is not included in the bit stream.
`SvtAv1EncApp.exe -c Sample.cfg -i CrowdRun_1920x1080.yuv -w 1920 -h 1080 -n 100 -q 30 --keyint 31 -b CrowdRun_1920x1080_qp30.bin`
It should be noted that not all the encoder parameters present in the `Sample.cfg` can be changed using the command line.
### Sample command lines
Here are some sample encode command lines
#### 1 pass CRF at maximum speed from 24fps yuv 1920x1080 input
`SvtAv1EncApp -i input.yuv -w 1920 -h 1080 --fps 24 --crf 30 --preset 8 -b output.ivf`
#### 1 pass VBR 1000 Kbps at medium speed from 24fps yuv 1920x1080 input
`SvtAv1EncApp -i input.yuv -w 1920 -h 1080 --fps 24 --rc 1 --tbr 1000 --preset 5 -b output.ivf`
#### 2 pass VBR 1000 Kbps at maximum quality from 24fps yuv 1920x1080 input
1 command line :
`SvtAv1EncApp -i input.yuv -w 1920 -h 1080 --fps 24 --rc 1 --tbr 1000 --preset 0 --irefresh-type 2 --passes 2 --stats stat_file.stat -b output.ivf`
or
2 command lines :
`SvtAv1EncApp -i input.yuv -w 1920 -h 1080 --fps 24 --rc 1 --tbr 1000 --preset 8 --irefresh-type 2 --pass 1 --stats stat_file.stat`
`SvtAv1EncApp -i input.yuv -w 1920 -h 1080 --fps 24 --rc 1 --tbr 1000 --preset 0 --irefresh-type 2 --pass 2 --stats stat_file.stat -b output.ivf`
### List of all configuration parameters
The encoder parameters present in the `Sample.cfg` file are listed in this table below along with their status of support, command line parameter and the range of values that the parameters can take.
#### Options
| **Configuration file parameter** | **Command line** | **Range** | **Default** | **Description** |
| --- | --- | --- | --- | --- |
| **InputFile** | -i | any string | None | Input file path; "stdin" to take input from standard input stream |
| **StreamFile** | -b | any string | null | Output bitstream file path; "stdout" to emit output to standard output stream |
| **ConfigFile** | -c | any string | null | Configuration file path |
| **ErrorFile** | --errlog | any string | stderr | error log displaying configuration or encode errors |
| **ReconFile** | -o | any string | null | Recon file path. Optional output of recon. |
| **StatFile** | --stat-file | any string | Null | Path to statistics file if specified and StatReport is set to 1, per picture statistics are outputted in the file|
| **Progress** | --progress | [0,1,2] | 1 | Use `--progress 0` to disable printing of frame processed when encoding, `--progress 1` for default printing, and `--progress 2` for aomenc style printing |
| **NoProgress** | --no-progress | [0,1] | 0 | `--no-progress 1` is equivalent to `--progress 0` and `--no-progress 0` is equivalent to `--progress 1` |
| **EncoderMode** | --preset | [-2,-1, 0 - 8] | 8 | Encoder Preset [-2, -1, 0,1,2,3,4,5,6,7,8] -2=debug preset, -1=debug preset 0 = highest quality, 8 = highest speed |
| **ChannelNumber** | --nch | [1 - 6] | 1 | Number of simultaneous encoding instances |
#### Encoder Global Options
| **Configuration file parameter** | **Command line** | **Range** | **Default** | **Description** |
| --- | --- | --- | --- | --- |
| **SourceWidth** | -w | [64 - 4096] | None | Input source width |
| **SourceHeight** | -h | [0 - 2304] | None | Input source height |
| **FrameToBeEncoded** | -n | [0 - 2^64 -1] | 0 | Number of frames to be encoded, if number of frames is > number of frames in file, the encoder will loop to the beginning and continue the encode. Use -1 to not buffer. |
| **BufferedInput** | --nb | [-1, 1 to 2^31 -1] | -1 | number of frames to preload to the RAM before the start of the encode If --nb = 100 and -n 1000 -- > the encoder will encode the first 100 frames of the video 10 times |
| **EncoderColorFormat** | --color-format | [0-3] | 1 | Set encoder color format(YUV400, YUV420, YUV422, YUV444 : YUV420 [default]) |
| **Profile** | --profile | [0-2] | 0 | Bitstream profile number to use (0: main profile[default], 1: high profile, 2: professional profile) |
| **FrameRate** | --fps | [0 - 2^64 -1] | 25 | If the number is less than 1000, the input frame rate is an integer number between 1 and 60, else the input number is in Q16 format (shifted by 16 bits) [Max allowed is 240 fps] |
| **FrameRateNumerator** | --fps-num | [0 - 2^64 -1] | 0 | Frame rate numerator e.g. 6000 |
| **FrameRateDenominator** | --fps-denom | [0 - 2^64 -1] | 0 | Frame rate denominator e.g. 100 |
| **EncoderBitDepth** | --input-depth | [8 , 10] | 8 | specifies the bit depth of the input video |
| **Encoder16BitPipeline** | --16bit-pipeline | [0 , 1] | 0 | Bit depth for enc-dec(0: lbd[default], 1: hbd) |
| **CompressedTenBitFormat** | --compressed-ten-bit-format | [0-1] | 0 | Offline packing of the 2bits: requires two bits packed input (0: OFF, 1: ON) |
| **HierarchicalLevels** | --hierarchical-levels | [0 - 5] | 4 | 0 : Flat4: 5-Level HierarchyMinigop Size = (2^HierarchicalLevels) (e.g. 0 == > 0B pyramid, 1 == > 1B pyramid, 2 == > 3B pyramid, 3 == > 7B pyramid, 4 == > 15B Pyramid) |
| **PredStructure** | --pred-struct | [0-2] | 2 | Set prediction structure( 0: low delay P, 1: low delay B, 2: random access [default]) |
| **Injector** | --inj | [0-1] | 0 | Inject pictures at defined frame rate(0: OFF[default],1: ON) |
| **InjectorFrameRate** | --inj-frm-rt | Null | Null | Set injector frame rate |
| **StatReport** | --enable-stat-report | [0-1] | 0 | When set to 1, calculates and outputs average PSNR values |
| **Asm** | --asm | [0 - 11] or [c, mmx, sse, sse2, sse3, ssse3, sse4_1, sse4_2, avx, avx2, avx512, max] | 11 or max | Limit assembly instruction set ("0" is equivalent to "c", "1" is "mmx" etc, max value is "11" or "max"), by default select highest assembly instruction that is supported by CPU |
| **LogicalProcessorNumber** | --lp | [0, total number of logical processor] | 0 | The number of logical processor which encoder threads run on.Refer to Appendix A.1 |
| **UnpinExecution** | --unpin | [0, 1] | 1 | Allows the execution to be pined/unpined to/from a specific number of cores.--unpin is overwritten to 0 when --ss is set to 0 or 1. 0=OFF, 1= ON |
| **TargetSocket** | --ss | [-1,1] | -1 | For dual socket systems, this can specify which socket the encoder runs on.Refer to Appendix A.1 |
#### Rate Control Options
| **Configuration file parameter** | **Command line** | **Range** | **Default** | **Description** |
| --- | --- | --- | --- | --- |
| **RateControlMode** | --rc | [0 - 1] | 0 | Rate control mode(0 = CQP if --enable-tpl-la is set to 0, else CRF , 1 = VBR)|
| **QP** | -q | [0 - 63] | 50 | Quantization parameter used when RateControl is set to 0 and EnableTPLModel is set to 0, also represents the CRF value if EnableTPLModel is set to 0 |
| **CRF** | --crf | [0 - 63] | 50 | Rate control parameter used to set CRF and forces RateControlMode to 0 and EnableTPLModel to 1 |
| **TargetBitRate** | --tbr | [1 - 4294967] | 7000 | Target bitrate in kilobits per second when RateControlMode is set to 1, or 2 |
| **UseQpFile** | --use-q-file | [0-1] | 0 | When set to 1, overwrite the picture qp assignment using qp values in QpFile, can be used for initial crf values as well if EnableTPLModel is set to 1, the encoder may still change crf per block |
| **QpFile** | --qpfile | any string | Null | Path to qp file |
| **MaxQpAllowed** | --max-qp | [0 - 63] | Null | Maximum (worst) quantizer[0-63] only applicable when --rc > 0 |
| **MinQpAllowed** | --min-qp | [0 - 63] | Null | Minimum (best) quantizer[0-63] only applicable when --rc > 0 |
| **AdaptiveQuantization** | --adaptive-quantization | [0 - 2] | 2 | 0 = OFF , 1 = variance base using segments , 2 = Deltaq pred efficiency (default) |
| **VBVBufSize** | --vbv-bufsize | [1 - 4294967] | 1 second TargetBitRate | VBV Buffer Size when RateControl is 2. |
| **UseFixedQIndexOffsets** | --use-fixed-qindex-offsets | [0 - 1] | 0 | 0 = OFF, 1 = enable fixed qindex offset based on temporal layer and frame type when rc mode is 0. qindex offsets are specified by the following arguments |
| **QIndexOffsets** | --qindex-offsets | [v0,v1,..,vn] | [0,0,..,0] | list of qindex offsets vi, enclosed in [], seperated by ,. vi is in the range of [-256,255]. this argument should be used after hierarchical-levels, the number of qindex offsets equals to hierarchical-levels + 1 |
| **KeyFrameQIndexOffset** | --key-frame-qindex-offset | [-256, 255] | 0 | qindex offset for Key frame |
| **ChromaQIndexOffsets** | --chroma-qindex-offsets | [v0,v1,..,vn] | [0,0,..,0] | list of qindex offset vi, enclosed in [], seperated by ,. vi is in the range of [-256, 255]. this argument should be used after hierarchical-levels, the number of qindex offsets equals to hierarchical-levels + 1 |
| **KeyFrameChromaQIndexOffset** | --key-frame-chroma-qindex-offset | [-256, 255] | 0 | chroma qindex offset for Key frame |
use-fixed-qindex-offsets and associtated arguments (hierarchical-levels, qindex-offsets, chroma-qindex-offsets, key-frame-qindex-offset, key-frame-chroma-qindex-offset) are used together to specify the qindex offsets based on frame type and temporal layer when rc mode = 0.
qp value specified by -q argument is assigned to the pictures at the highest temporal layer. it is first converted to qindex, then corresponding qindex offsets are added on top of it based on frame types (Key/Inter) and temporal layer id.
qindex offset can be negative. the final qindex value will be clamped in the valid min/max qindex range.
for chroma plane, after deciding the qindex for the luma plane, corresponding chroma qindex offsets are added on top of the luma plane qindex based on frame types and temporal layer id.
--qindex-offsets and --chroma-qindex-offsets have to be used after the --hierachical-levels parameter. number of qindex offsets should be hierachical levels plus 1.
an example command line is:
"--rc 0 -q 42 --hierarchical-levels 3 --use-fixed-qindex-offsets 1 --qindex-offsets [-12,-8,-4,0] --key-frame-qindex-offset -20 --key-frame-chroma-qindex-offset -6 --chroma-qindex-offsets [-6,0,12,24]"
for this command line, corresponding qindex are:
| **Frame Type** | **Luma qindex** | **Chroma qindex** |
| --- | --- | --- |
| **Key Frame** | 148 (42x4 - 20) | 142 (148 - 6) |
| **Layer0 Frame** | 156 (42x4 - 12) | 150 (156 - 6) |
| **Layer1 Frame** | 160 (42x4 - 8) | 160 (160 + 0) |
| **Layer2 Frame** | 164 (42x4 - 4) | 176 (164 + 12) |
| **Layer3 Frame** | 168 (42x4 + 0) | 192 (168 + 24) |
#### Twopass Options
| **Configuration file parameter** | **Command line** | **Range** | **Default** | **Description** |
| --- | --- | --- | --- | --- |
| **Passes** | --passes | [1-2] | 1 | Number of passes (1: one pass encode, 2: two passes encode) applicable only for rc > 0 |
| **Pass** | --pass | [1-2] | Null | Specify which pass the run is on (1=First Pass, 2=Second Pass) |
| **Stats** | --stats | any string | Null | Output stat file containing information from first pass |
| **OutputStatFile** | --output-stat-file | any string | Null | Output stat file for first pass|
| **InputStatFile** | --input-stat-file | any string | Null | Input stat file for second pass|
| **VBRBiasPct** | --bias-pct | [0 - 100] | 50 | 2pass CBR/VBR bias percent (0=CBR-like, 100=VBR-like) |
| **MinSectionPct** | --minsection-pct | [0 - ] | 0 | 2pass VBR GOP min bitrate (percent of target) |
| **MaxSectionPct** | --maxsection-pct | [0 - ] | 2000 | 2pass VBR GOP max bitrate (percent of target) |
| **UnderShortPct** | --undershoot-pct | [0 - 100] | 25 | Datarate undershoot (min) target (percent) |
| **OverShortPct** | --overshoot-pct | [0 - 100] | 25 | Datarate overshoot (max) target (percent) |
| **RecodeLoop** | --recode-loop | [0 - 4] | 4 | Recode loop levels for 2pass VBR (0=disable reencode, 1=reencode key frames, 2=reencode base layer frames, 3=reencode all frames, 4=preset based decision [Default]) |
#### GOP size and type Options
| **Configuration file parameter** | **Command line** | **Range** | **Default** | **Description** |
| --- | --- | --- | --- | --- |
| **IntraPeriod** | --keyint | [-2 - 2^31-2] | -2 | Intra period interval(frames) -2: default intra period , -1: No intra update or an integer >= 0. if RateControlMode >= 1 intra-period limited to [-2, 255] |
| **IntraRefreshType** | --irefresh-type | [1 - 2] | 2 | Intra refresh type (1: FWD Frame (Open GOP), 2: KEY Frame (Closed GOP)[default]) |
#### AV1 Specific Options
| **Configuration file parameter** | **Command line** | **Range** | **Default** | **Description** |
| --- | --- | --- | --- | --- |
| **TileRow** | --tile-rows | [0-6] | 0 | log2 of tile rows |
| **TileCol** | --tile-columns | [0-4] | 0 | log2 of tile columns |
| **LoopFilterDisable** | --disable-dlf | [0-1] | 0 | Disable loop filter(0: loop filter enabled[default] ,1: loop filter disabled) |
| **EnableTPLModel** | --enable-tpl-la | [0-1] | 1 | RDO based on frame temporal dependency (0: off, 1: backward source based)|
| **CDEFLevel** | --cdef-level | [0-5] | -1 | CDEF Level, 0: OFF, 1-5: ON with 64,16,8,4,1 step refinement, -1: DEFAULT|
| **RestorationFilter** | --enable-restoration-filtering | [0-1] | -1 | Enable restoration filtering , 0 = OFF, 1 = ON, -1 = DEFAULT|
| **SelfGuidedFilterMode** | --sg-filter-mode | [0-4] | -1 | Self-guided filter mode (0:OFF, 1: step 0, 2: step 1, 3: step 4, 4: step 16, -1: DEFAULT)|
| **WienerFilterMode** | --wn-filter-mode | [0-3] | -1 | Wiener filter mode (0:OFF, 1: 3-Tap luma/ 3-Tap chroma, 2: 5-Tap luma/ 5-Tap chroma, 3: 7-Tap luma/ 7-Tap chroma, -1: DEFAULT)|
| **Mfmv** | --enable-mfmv | [0-1] | -1 | Enable motion field motion vector, 0 = OFF, 1 = ON, -1 = DEFAULT|
| **RedundantBlock** | --enable-redundant-blk | [0-1] | -1 | Enable redundant block skipping same neighbors non-square partitions, 0 = OFF, 1 = ON, -1 = DEFAULT|
| **SpatialSSEfl** | --enable-spatial-sse-full-loop-level | [0-1] | -1 | Enable spatial sse full loop, 0 = OFF, 1 = ON, -1 = DEFAULT|
| **OverBoundryBlock** | --enable-over-bndry-blk | [0-1] | -1 | Enable over boundary block mode, 0 = OFF, 1 = ON, -1 = DEFAULT|
| **NewNearestCombInjection** | --enable-new-nrst-near-comb | [0-1] | -1 | Enable new nearest near comb injection, 0 = OFF, 1 = ON, -1 = DEFAULT|
| **NsqTable** | --enable-nsq-table-use | [0-1] | -1 | Enable nsq table, 0 = OFF, 1 = ON, -1 = DEFAULT|
| **FrameEndCdfUpdate** | --enable-framend-cdf-upd-mode | [0-1] | -1 | Enable frame end cdf update mode, 0 = OFF, 1 = ON, -1 = DEFAULT|
| **ChromaMode** | --chroma-mode | [0-3] | -1 | Chroma Mode
-1 = DEFAULT
0 = Full chroma search @ MD
1 = Fast chroma search @ MD
2 = Chroma blind @ MD + CFL @ EP
3 = Chroma blind @ MD + no CFL @ EP |
| **DisableCfl** | --disable-cfl | [0-1] | -1 | Disable chroma from luma (CFL), 0 = OFF (do not disable), 1 = ON (disable), -1 = DEFAULT|
| **LocalWarpedMotion** | --enable-local-warp | [0-1] | -1 | Enable warped motion use , 0 = OFF, 1 = ON, -1 DEFAULT |
| **GlobalMotion** | --enable-global-motion | [0-1] | 1 | Enable global motion (0: OFF, 1: ON [default]) |
| **PicBasedRateEst** | --enable-pic-based-rate-est | [0-1] | -1 | Enable picture based rate estimation. Only active with lp 1 (0: OFF, 1: ON, -1: DEFAULT)|
| **IntraAngleDelta** | --enable-intra-angle-delta | [0-1] | -1 | Enable intra angle delta filtering (0: OFF, 1: ON, -1 = DEFAULT |
| **InterIntraCompound** | --enable-interintra-comp | [0-1] | -1 | Enable inter intra compound, 0 = OFF, 1 = ON, -1 = DEFAULT|
| **Paeth** | --enable-paeth | [0-1] | -1 | Enable Intra Paeth pred, 0 = OFF, 1 = ON, -1 = DEFAULT|
| **Smooth** | --enable-smooth | [0-1] | -1 | Enable Intra Smooth pred, 0 = OFF, 1 = ON, -1 = DEFAULT|
| **MultiReferencePictures** | --mrp-level | [0-9] | -1 | Multi reference frame levels( 0: OFF, 1: FULL, 2: Level1 .. 9: Level8, -1: DEFAULT)|
| **Obmc** | --obmc-level | [0-3] | -1 | OBMC Level(0: OFF, 1: Fully ON, 2 and 3 are faster levels, -1: DEFAULT) |
| **RDOQ** | --rdoq-level | [0-1] | -1 | Enable RDOQ, 0 = OFF, 1 = ON, -1 = DEFAULT |
| **FilterIntra** | --filter-intra-level | [0-1] | -1 | Enable filter intra prediction mode (0: OFF, 1: ON [default]) |
| **IntraEdgeFilter** | --enable-intra-edge-filter | [0-1] | -1 | Enable intra edge filter (0: OFF, 1: ON, -1: DEFAULT)|
| **PredMe** | --pred-me | [0-5] | -1 | Closed loop motion search. Set predictive me level:
-1 = DEFAULT
0 = OFF
1 = 7x5 full-pel search + sub-pel refinement off
2 = 7x5 full-pel search + (H + V) sub-pel refinement only = 4 half-pel + 4 quarter-pel = 8 positions + pred_me_distortion to pa_me_distortion deviation on
3 = 7x5 full-pel search + (H + V + D only ~ the best) sub-pel refinement = up to 6 half-pel + up to 6 quarter-pel = up to 12 positions + pred_me_distortion to pa_me_distortion deviation on
4 = 7x5 full-pel search + (H + V + D) sub-pel refinement = 8 half-pel + 8 quarter-pel = 16 positions + pred_me_distortion to pa_me_distortion deviation on
5 = 7x5 full-pel search + (H + V + D) sub-pel refinement = 8 half-pel + 8 quarter-pel = 16 positions + pred_me_distortion to pa_me_distortion deviation off |
| **Bipred3x3** | --bipred-3x3 | [0-2] | -1 | Set bipred3x3 injection, 0 = OFF, 1 = ON FULL, 2 = Reduced set, -1 = DEFAULT|
| **CompoundLevel** | --compound | [0-2] | -1 | Set compound mode:
-1 = DEFAULT
0 = OFF: No compond mode search : AVG only
1 = ON: compond mode search: AVG/DIST/DIFF
2 = ON: AVG/DIST/DIFF/WEDGE |
| **UseDefaultMeHme** | --use-default-me-hme | [0-1] | 1 | 0 : Overwrite Default ME HME parameters1 : Use default ME HME parameters, dependent on width and height |
| **HME** | --hme | [0-1] | 1 | Enable HME, 0 = OFF, 1 = ON |
| **HMELevel0** | --hme-l0 | [0-1] | 1 | Enable HME Level 0 , 0 = OFF, 1 = ON |
| **HMELevel1** | --hme-l1 | [0-1] | Depends on input resolution | Enable HME Level 1 , 0 = OFF, 1 = ON |
| **HMELevel2** | --hme-l2 | [0-1] | Depends on input resolution | Enable HME Level 2 , 0 = OFF, 1 = ON |
| **ExtBlockFlag** | --ext-block | [0-1] | Depends on --preset | Enable the non-square block 0=OFF, 1= ON |
| **SearchAreaWidth** | --search-w | [1 - 480] | Depends on input resolution | Search Area in Width |
| **SearchAreaHeight** | --search-h | [1 - 480] | Depends on input resolution | Search Area in Height |
| **ScreenContentMode** | --scm | [0 - 2] | 2 | Enable Screen Content Optimization mode (0: OFF, 1: ON, 2: Content Based Detection) |
| **IntraBCMode** | --intrabc-mode | [-1 - 3]] | -1 | IntraBC mode (0 = OFF, 1 = ON slow, 1 = ON faster, 2 = ON fastest, -1 = DEFAULT) |
| **HighBitDepthModeDecision** | --hbd-md | [0-2] | 1 | Enable high bit depth mode decision(0: OFF, 1: ON partially[default],2: fully ON) |
| **PaletteLevel** | --palette-level | [-1 - 6] | -1 | Enable Palette mode (-1: DEFAULT (ON at level6 when SC is detected), 0: OFF 1: ON Level 1, ...6: ON Level6 ) |
| **UnrestrictedMotionVector** | --umv | [0-1] | 1 | Enables or disables unrestriced motion vectors, 0 = OFF(motion vectors are constrained within tile boundary), 1 = ON. For MCTS support, set --umv 0 |
| **FilmGrain** | --film-grain | [0-50] | 0 | Enable film grain(0: OFF[default], 1 - 50: Level of denoising for film grain) |
| **AltRefLevel** | --tf-level | [-1 - 1] | -1 | Enable automatic alt reference frames(-1: Default; 0: OFF; 1: ON) |
| **EnableOverlays** | --enable-overlays | [0-1] | 0 | Enable the insertion of an extra picture called overlayer picture which will be used as an extra reference frame for the base-layer picture(0: OFF[default], 1: ON) |
| **SquareWeight** | --sqw | 0 for off and any whole number percentage | 100 | Weighting applied to square/h/v shape costs when deciding if a and b shapes could be skipped. Set to 100 for neutral weighting, lesser than 100 for faster encode and BD-Rate loss, and greater than 100 for slower encode and BD-Rate gain|
the DEFAULT option would allow the encoder to choose adaptively any of the values in the range for that option whether on a preset, picture, or SB level.
| **ChannelNumber** | --nch | [1 - 6] | 1 | Number of encode instances |
| **StatReport** | --enable-stat-report | [0-1] | 0 | When set to 1, calculates and outputs average PSNR values |
| **ColorPrimaries** | --color-primaries | [0-12, 22] | 2 | Set color primaries, please see the subsection 6.4.2 of the [AV1 Bitstream & Decoding Process Specification](https://aomediacodec.github.io/av1-spec/av1-spec.pdf) for details |
| **TransferCharacteristics** | --transfer-characteristics | [0-22] | 2 | Set transfer characteristics, please see the subsection 6.4.2 of the [AV1 Bitstream & Decoding Process Specification](https://aomediacodec.github.io/av1-spec/av1-spec.pdf) for details |
| **MatrixCoefficients** | --matrix-coefficients | [0-14] | 2 | Set matrix coefficients, please see the subsection 6.4.2 of the [AV1 Bitstream &; Decoding Process Specification](https://aomediacodec.github.io/av1-spec/av1-spec.pdf) for details |
| **ColorRange** | --color-range | [0-1] | 0 | Set color range, please see the subsection 6.4.2 of the [AV1 Bitstream &; Decoding Process Specification](https://aomediacodec.github.io/av1-spec/av1-spec.pdf) for details |
## Appendix A Encoder Parameters
### 1. Thread management parameters
LogicalProcessorNumber (`--lp`) and TargetSocket (`--ss`) parameters are used to management thread affinity on Windows and Ubuntu OS. These are some examples how you use them together.
If LogicalProcessorNumber and TargetSocket are not set, threads are managed by OS thread scheduler.
`SvtAv1EncApp.exe -i in.yuv -w 3840 -h 2160 --lp 40`
If only LogicalProcessorNumber is set, threads run on 40 logical processors. Threads may run on dual sockets if 40 is larger than logical processor number of a socket.
NOTE: On Windows, thread affinity can be set only by group on system with more than 64 logical processors. So, if 40 is larger than logical processor number of a single socket, threads run on all logical processors of both sockets.
`SvtAv1EncApp.exe -i in.yuv -w 3840 -h 2160 --ss 1`
If only TargetSocket is set, threads run on all the logical processors of socket 1.
`SvtAv1EncApp.exe -i in.yuv -w 3840 -h 2160 --lp 20 --ss 0`
If both LogicalProcessorNumber and TargetSocket are set, threads run on 20 logical processors of socket 0. Threads guaranteed to run only on socket 0 if 20 is larger than logical processor number of socket 0.
The (`-unpin`) option allows the user to pin/unpin the execution to/from a specific number of cores.
The combinational use of (`-unpin`) with (`-lp`) results in memory reduction while allowing the execution to work on any of the cores and not restrict it to specific cores.
This is an example on how to use them together.
so -lp 4 with -unpin 0 would restrict the encoder to work on cpu0-3 and reduce the resource allocation to only what's needed to using 4 cores. -lp 4 with -unpin 1, would reduce the allocation to what's needed for 4 cores but not restrict the encoder to run on cpu 0-3, in this case the encoder might end up using more than 4 cores due to the multi-threading nature of the encoder, but would at least allow for more multiple -lp4 encodes to run on the same machine without them being all restricted to run on cpu 0-3 or overflow the memory usage.
Example: 72 core machine:
72 jobs x --lp 1 --unpin 1 (In order to maximize the CPU utilization 72 jobs are run simultaneously with each job utilitizing 1 core without being pined to a specific core)
36 jobs x --lp 2 --unpin 1
18 jobs x --lp 4 --unpin 1
(`-ss`) and (`-unpin 1`) is not a valid combination.(`-unpin`) is overwritten to 0 when (`-ss`) is used.
PK #2]1)Od d Appendix-CDEF.mdnu [ # Constrained Directional Enhancement Filter (CDEF) Appendix
## 1. Description of the algorithm
The constrained directional enhancement filter (CDEF) is applied after
the deblocking filter and aims at improving the reconstructed picture by
addressing ringing artifacts. CDEF is a combination of the directional
de-ringing filter from the Daala codec (Mozilla) and the Constrained Low
Pass Filter (CLPF) from the Thor codec (Cisco).
Filtering is applied on an 8x8 block level, which is a large enough
block size to reliably detect edges, but small enough block size to
accurately detect edge directions. Filtering is applied to both luma and
chroma samples. For a given block, the algorithm consists of the two
main steps outlined below:
1. Identify the direction **d** of the block (i.e. direction of edges). Eight directions {0,…,7} could be identified.
2. Filtering
* Apply a nonlinear filter along the edge in the identified direction. Filter taps are aligned in the direction of the block. The main goal is to address ringing artifacts.
* Filter mildly along a 45 degrees direction from the edge.
The two steps are outlined in more detail in the following.
***Step 1 – Identification of edge direction***. Eight edge directions
could be considered. The directions are indexed with d=0,…,7 as
indicated in Figure 1 below.

##### Figure 1. Block directions.
For a given input block, the identification of edge direction is
performed as follows:
- Input pixels along each of the lines k=0,1,… are averaged. The
original sample values along each on the lines k=0,1,… are replaced
by the average value of the samples on the line. The resulting block
is referred to as the averaged block.
- The variance of the error between the source block and the averaged
block is computed.
- The operations above are repeated for each of the eight directions.
- The direction with the lowest variance is selected as the filtering
direction.
The example in Figure 2 below illustrates this step for an 8x8 input block. In this
example, direction 0 resulted in the smallest error variance and hence
was selected as the block direction.

##### Figure 2. Example of block direction identification.
***Step 2 – Filtering***. The filtering operation consists of two main
steps, namely a primary filtering operation and a secondary filtering
operation. The primary filter acts along the identified block direction.
The secondary filter acts at  from the identified
direction. In the example shown in Figure 3 below, the block direction
is d=0 . The sample to be filtered is highlighted in
red. The samples to be considered when filtering the red sample in
primary filtering are highlighted in green (a total of four samples).
Those considered in the secondary filtering of the red sample are
located at  angle from the block direction and are
highlighted in blue (a total of eight samples).

##### Figure 3. Example of primary and secondary filtering directions.
The nonlinear low-pass filters used in the filtering process are given
by [\[1\]](#ref-1):
=p(i,j)+\sum_{m,n}w_{m,n}f(p(m,n)-p(i,j),S,D))
Where ) is the sample to be filtered,
) is the filtered value of
sample ),  are the filter weights,
 is a nonlinear constraint function,  is the
filter strength and  is the filter damping.
When -p(i,j)|) is small, -p(i,j),S,D)=p(m,n)-p(i,j)), implying that the filter
behaves as an FIR filter. When -p(i,j)|) is large,
-p(i,j),S,D)=0) and no filtering is applied to the sample. The function 
de-emphasizes neighboring pixels ) with large
contrast (i.e. large magnitude of -p(i,j))). The strength
 and damping  parameters control the attenuation of the large magnitude differences.
Filtering along the identified block direction is referred to as primary
filtering, and makes use of primary filtering strength and primary
damping values. The filter weights for primary filtering are shown in
Figure 4 below. The sample to be filtered is shown in blue. For even
strengths, a = 2 and b = 4, whereas for odd strengths a = 3 and b = 3.

##### Figure 4. Filter weights for primary filtering.
Filtering of samples that are at 45 degrees from the identified edge
direction for the block is referred to as secondary filtering. Secondary
filtering makes use of secondary strength and secondary damping values.
The filter weights for secondary filtering are indicated in Figure 5
below as a function of the block direction. The sample to be filtered is
shown in blue.

##### Figure 5. Filter weights for secondary filtering.
## 2. Implementation
**Inputs to cdef\_kernel**: Output frame from the deblocking filter.
**Outputs of cdef\_kernel**: CDEF filtered frame, filter parameters.
**Controlling macros/flags**:
Control flags associated with CDEF are listed in Table 1 below.
##### Table 1. Control flags for CDEF.
| **Flag** | **Level** | **Description** |
| ------------------------------- | -------------- | -------------------------------------------------------------------------------------------------------------------------- |
| -cdef-level | Configuration | Command line option: 0: OFF, 1-5: ON with steps 64,16,8,4,1, -1: Auto mode (determined in code) |
| cdef\_level | Sequence | Indicates whether to use CDEF for the whole sequence. |
| cdef\_level | Picture | Indicates the level of complexity of the CDEF strength search as a function of the encoder mode (enc\_mode). |
| use\_ref\_frame\_cdef\_strength | Picture | If set, use the CDEF strength for the reference frame in optimizing the search for the CDEF strength in the current frame. |
**Implementation details**
Important function calls associated with CDEF are highlighted in Figure 6 below. The function calls are
organized according to the depth of the function call.

##### Figure 6. The main function calls associated with CDEF.
The main steps involved in the implementation of the algorithm are
outlined below, followed by more details on some of the important
functions.
Step 1 - Splitting the frame into segments
The frame to be filtered is divided into segments to allow for parallel
filtering operations on different parts of the frame. The segments are
set according to the following (see EbEncHandle.c)
```c
uint32_t meSegH = (((sequence_control_set_ptr->max_input_luma_height + 32) / BLOCK_SIZE_64) < 6) ? 1 : 6;
uint32_t meSegW = (((sequence_control_set_ptr->max_input_luma_width + 32) / BLOCK_SIZE_64) < 10) ? 1 : 10;
sequence_control_set_ptr->cdef_segment_column_count = meSegW;
sequence_control_set_ptr->cdef_segment_row_count = meSegH;
```
The segments are processed in ```cdef_kernel```. Each segment is split into
64x64 filter blocks.
Step 2: Perform CDEF search for each segment \[each running on a
separate thread\]. Each segment goes through a filter search operation
through the function (```cdef_seg_search```). For a given 64x64 filter block
in a segment, the main purpose of the search is to identify the
directions of the 8x8 blocks in the filter block, and the best filter
(Primary strength, Secondary strength) pair to use in filtering the
filter block. The primary filter strength takes value in {0,…,15},
whereas the secondary filter strength takes value in {0, 1, 2, 4}. The
(primary strength, secondary strength) pairs are then indexed and
ordered as indicated in Table 2 below:
##### Table 2. (primary strength, secondary strength) pairs.
| **Filter Strength Index** | **(Primary Strength, Secondary Strength) Pair** |
| ------------------------- | ----------------------------------------------- |
| 0 | (0,0) |
| 1 | (0,1) |
| 2 | (0,2) |
| 3 | (0,4) |
| 4 | (1,0) |
| 5 | (1,1) |
| … | (...,…) |
| 63 | (15,4) |
The search for the best (Primary strength, Secondary strength) pair to
use is equivalent to the search for the index for such pair.
The primary and secondary luma damping values are set as a function of
the base qindex for the picture and are given by:
```c
pri_damping = 3 + (picture_control_set_ptr->parent_pcs_ptr->base_qindex >> 6);
sec_damping = 3 + (picture_control_set_ptr->parent_pcs_ptr->base_qindex >> 6);
```
Chroma damping values are always one less the luma damping value.
The CDEF search proceeds along the following steps.
- Loop over all 64x64 filter blocks in the segment.
- Loop over the picture planes
- Set the range of filter strength index \[`start_gi`, `end_gi`\] to
be considered in the best filter strength search. Details on how
to set `start_gi` and `end_gi` are provided in the optimization
section below.
- Loop over the filter strengths in the filter strength index
range \[`start_gi`, `end_gi`\]
- Perform the following for each 8x8 non-skip block
(`cdef_filter_fb`):
- Find the direction for each 8x8 block (`cdef_find_dir`).
- Filter the 8x8 block according to the identified
direction using the set filter strength
(`cdef_filter_block`, C only version
`cdef_filter_block_c`. More details on
`cdef_filter_block_c` are provided below.).
- Compute the filtering mse for the filter block corresponding
to the filter strength being considered
(`compute_cdef_dist`).
Step 3: Select a subset of filter strengths to use in the final
filtering of the 64x64 filter blocks in the frame based on the filtering
results from step 2 (`finish_cdef_search`. More details on
`finish_cdef_search` are provided below.). This step is frame-based, and is performed by
only one thread.
Step 4: Complete the filtering of the frame based on the selected set of
filtering strengths from Step 3. (`av1_cdef_frame`. More details on
`av1_cdef_frame` are provided below.)
**More details about cdef\_filter\_block\_c**
For a given 8x8 block, filtering is applied to all samples in the 8x8
block. Filtering is to be applied according to the identified direction
for the 8x8 block. For a given sample to be filtered in the block, the
position of the neighboring samples to be considered in the filtering
operation are given by the array `cdef_directions` according to the
identified direction. The array is given by
```c
DECLARE_ALIGNED(16, const int32_t, cdef_directions[8][2]) = {
{ -1 * CDEF_BSTRIDE + 1, -2 * CDEF_BSTRIDE + 2 },
{ 0 * CDEF_BSTRIDE + 1, -1 * CDEF_BSTRIDE + 2 },
{ 0 * CDEF_BSTRIDE + 1, 0 * CDEF_BSTRIDE + 2 },
{ 0 * CDEF_BSTRIDE + 1, 1 * CDEF_BSTRIDE + 2 },
{ 1 * CDEF_BSTRIDE + 1, 2 * CDEF_BSTRIDE + 2 },
{ 1 * CDEF_BSTRIDE + 0, 2 * CDEF_BSTRIDE + 1 },
{ 1 * CDEF_BSTRIDE + 0, 2 * CDEF_BSTRIDE + 0 },
{ 1 * CDEF_BSTRIDE + 0, 2 * CDEF_BSTRIDE - 1 }
};
```
The primary and secondary filter coefficients are given by the array:
```c
const int32_t cdef_pri_taps[2][2] = { { 4, 2 }, { 3, 3 } };
const int32_t cdef_sec_taps[2][2] = { { 2, 1 }, { 2, 1 } };
```
where `cdef_pri_taps[0]` correspond to the case of even primary
strength, whereas `cdef_pri_taps[1]` correspond to the case of odd
primary strength.
**More details on finish\_cdef\_search in step 3**
For each 64x64 filter block, the output from step 2 is an array of
distortion values corresponding to different filter strength pairs
(Primary strength, Secondary strength). To reduce the overhead
associated with the signaling of the individual filter strength index
for each 64x64 filter block, only the a subset of the identified filter
strength pairs is selected. Final filtering of the 64x64 filter blocks
in the frame is then redone using the best among the selected subset of
filter strengths. The encoder needs to signal to decoder only the
selected subset of filter strengths for the decoder to use in the
filtering operation. The encoder could signal a set that consists of
only 1, 2, 4, or 8 different (Primary strength, Secondary strength)
pairs to be used for the frame. The specific pair to use for each 64x64
filter block is signaled separately. The search performed in
`finish_cdef_search` is to find the best RDO option (i.e. 1, 2, 4, or 8
filter strength pairs for the frame) to work with.
- Loop over the cardinality of the set of the strength pair options (1
then 2 then 4 then 8)
- Call the function `joint_strength_search_dual` to determine the
best such set for each of the options based on filtering distortion
(See below for details on the `joint_strength_search_dual` function
that makes use of a greedy search algorithm). Compute the RDO cost
of each of the options and keep track of the best option (i.e. the
best number of bits and the corresponding set of best (Primary
strength, Secondary strength) pairs. The latter are stored in the
following arrays:
```c
for (j = 0; j < 1 << nb_strength_bits; j++) {
pPcs->cdef_strengths[j] = best_lev0[j];
pPcs->cdef_uv_strengths[j] = best_lev1[j];
}
```
- Loop over the filter blocks in the frame and select for each filter
block the best (Primary strength, Secondary strength) pair. The
selected pair is signaled in
```c
picture_control_set_ptr->mi_grid_base[sb_index[i]]->mbmi.cdef_strength = (int8_t)best_gi;
```
- Damping is stored in
```c
pPcs->cdef_pri_damping = pri_damping;
pPcs->cdef_sec_damping = sec_damping;
```
- The most used filter strength pair in the filter blocks for the
frame is then identified as the frame strength and its corresponding
index is stored in `pPcs`->`cdef_frame_strength`.
**More details on joint\_strength\_search\_dual**
**Inputs:**
```c
*best_lev0, /* Array of indices of selected luma strengths. */
*best_lev1, /* Array of indices of selected chroma strengths. */
nb_strengths, /* Number of selected (Luma_strength, Chroma_strength) pairs */
(**mse)[TOTAL_STRENGTHS], /* Array of luma and chroma filtering mse values */
sb_count, /* Number of filter blocks in the frame */
fast, /* A flag indicating if fast search is to be performed */
start_gi, /* starting strength index for the search of the additional strengths */
end_gi /* End index for the for the search of the additional strengths */
/* Determine nb_strengths (Luma_strength, Chroma_strength) pairs.
The list of nb_strengths pairs is determined by adding one such pair at
a time through the call to the function search_one_dual. When the
function search_one_dual is called, the search accounts for the
strength pairs that have already been added in the previous iteration of
the loop below. The loop below returns in the end best_tot_mse
representing the best filtering mse for the whole frame based on the
selected list of best (Luma_strength, Chroma_strength) pairs. */
for i = 0 to i = (nb_strengths-1) do
best_tot_mse = search_one_dual(best_lev0, best_lev1, i, mse, sb_count, fast, start_gi, end_gi);
/* Performing further refinements on the search based on the results
from the step above. */
for i = 0 to i = (4 * nb_strengths -1) do
for j = 0 to j = (nb_strengths – 2) do
best_lev0[j] = best_lev0[j + 1];
best_lev1[j] = best_lev1[j + 1];
end for
best_tot_mse = search_one_dual(best_lev0, best_lev1, nb_strengths - 1, mse, sb_count, fast, start_gi, end_gi);
end for
return best_tot_mse;
```
**More details on search\_one\_dual\_c**
**Inputs:**
```c
*lev0 /* Array of indices of selected luma strength. */
*lev1, /* Array of indices of selected chroma strength. */
nb_strengths, /* Number of selected (Luma_strength, Chroma_strength) pairs */
(**mse)[TOTAL_STRENGTHS], /* Array of luma and chroma filtering mse values */
sb_count, /* Number of filter blocks in the frame */
fast, /* A flag indicating if fast search is to be performed*/
start_gi, /* starting strength index for the search of the additional strengths */
end_gi /* End index for the for the search of the additional strengths */
best_tot_mse = 1 << 63;
total_strengths = end_gi;
memset(tot_mse, 0, sizeof(tot_mse));
for i = 0 to (sb_count-1) do /*Loop over the filter blocks in the frame */
best_mse = 1<< 63;
/* Loop over the already selected nb_strengths (Luma_strength,
Chroma_strength) pairs, and find the pair that has the smallest mse
(best_mse) for the current filter block. */
for gi = 0 to gi = (nb_strengths-1) do
curr = mse[0][i][lev0[gi]];
curr += mse[1][i][lev1[gi]];
if (curr < best_mse) then best_mse = curr;
end for
/* Loop over the set of available (Luma_strength, Chroma_strength)
pairs, identify any that provide an mse better than best_mse from the
step above for the current filter block, and update any corresponding
total mse (tot_mse[j][k]). */
for j = start_gi; j = (total_strengths-1) do
for k = start_gi; k = (total_strengths-1) do
best = best_mse
curr = mse[0][i][j] + mse[1][i][k]
if (curr < best) best = curr
tot_mse[j][k] += best;
end for
end for
end for
/* Loop over the additionally searched (Luma_strength, Chroma_strength) pairs
from the step above, and identify any such pair that provided the best mse for
the whole frame. The identified pair would be added to the set of already selected pairs. */
for j = start_gi to j = (total_strengths-1) do /* Loop over the additionally searched luma strengths */
for k = start_gi to k = (total_strengths-1) do /* Loop over the additionally searched chroma strengths */
if (tot_mse[j][k] < best_tot_mse)
best_tot_mse = tot_mse[j][k];
best_id0 = j; /* index for the best luma strength */
best_id1 = k; /* index for the best chroma strength */
end if
end for
end for
lev0[nb_strengths] = best_id0; /* Add the identified luma strength to the list of selected luma strengths */
lev1[nb_strengths] = best_id1; /* Add the identified chroma strength to the list of selected chroma strengths */
return best_tot_mse;
```
**More details on av1\_cdef\_frame**
Loop over the 64x64 filter blocks.
```c
/*Find the index of the CDEF strength for the filter block*/
mbmi_cdef_strength = pCs->mi_grid_base[MI_SIZE_64X64 * fbr * cm->mi_stride + MI_SIZE_64X64 * fbc]->mbmi.cdef_strength;
/*Primary luma strength*/
level = pCs->parent_pcs_ptr->cdef_strengths[mbmi_cdef_strength] / CDEF_SEC_STRENGTHS;
/*secondary luma strength*/
sec_strength = pCs->parent_pcs_ptr->cdef_strengths[mbmi_cdef_strength] % CDEF_SEC_STRENGTHS;
/*secondary luma strength takes values in {0, 1, 2, 4}. If sec_strength is equal to 3 from the step above, change it to 4. */
sec_strength += sec_strength == 3;
/*Primary chroma strength*/
uv_level = pCs->parent_pcs_ptr->cdef_uv_strengths[mbmi_cdef_strength] / CDEF_SEC_STRENGTHS;
/*secondary chroma strength*/
uv_sec_strength = pCs->parent_pcs_ptr->cdef_uv_strengths[mbmi_cdef_strength] % CDEF_SEC_STRENGTHS;
/*secondary luma strength takes values in {0, 1, 2, 4}. If sec_strength is equal to 3 from the step above, change it to 4. */
uv_sec_strength += uv_sec_strength == 3;
```
Loop over the three picture planes
Call `cdef_filter_fb` to filter the samples in the filter block.
## 3. Optimization of the algorithm
The algorithmic optimization of the CDEF is performed by adjusting the
range of filter strength index used in the search for the best filter
strength pair for the 64x64 block. First, the encoder preset
(`picture_control_set_ptr`->`enc_mode`) is used to specify the CDEF
filter mode (`picture_control_set_ptr`->`cdef_level`) according
to Table 3 below.
##### Table 3. cdef_level as a function of encoder preset.
|**Encoder Preset (enc\_mode)** | **cdef\_level (Case of (slice\_type == I_SLICE) = 0)** | **cdef\_level (Case of (slice\_type == I_SLICE) = 1)** |
|--- |--- |--- |
|0|1|1|
|1|1|1|
|2|1|1|
|3|1|1|
|4|1|1|
|5|1|1|
|6|1|4|
|7|1|4|
|8|1|4|
The `cdef_level` specifies the parameter `gi_step` through the
function `get_cdef_gi_step`
`gi_step` = `get_cdef_gi_step`(`pPcs`->`cdef_level`);
`gi_step` represents half the width of the filter strength search
interval and is given in Table 4 below as a function of
`cdef_level`.
##### Table 4. gi_step as a function of cdef_levels.
| **cdef\_level** | **gi\_step** |
| ---------------------- | ------------ |
| **0** | OFF |
| **1** | 64 |
| **2** | 16 |
| **3** | 8 |
| **4** | 4 |
| **5** | 1 |
The search `in cdef_seg_search` and in `finish_cdef_search` for the
filter strength is performed by considering a sub-interval of the filter
strength index \[0,63\]. The subinterval is given by
\[`cdef_ref_frame_strength` – `gi_step`, `cdef_ref_frame_strength` +
`gi_step`\], where `cdf_ref_frame_strength` is the filter strength for
the reference picture. The actual implementation is given below:
```c
gi_step = get_cdef_gi_step(pPcs->cdef_level);
mid_gi = pPcs->cdf_ref_frame_strength;
start_gi = pPcs->use_ref_frame_cdef_strength && pPcs->cdef_level == 5 ? (AOMMAX(0, mid_gi - gi_step)) : 0;
end_gi = pPcs->use_ref_frame_cdef_strength ? AOMMIN(total_strengths, mid_gi + gi_step) : pPcs->cdef_level == 5 ? 8 : total_strengths;
```
4. **Signaling**
At the frame level, the algorithm signals the luma damping value and up
to 8 different filter strength presets to choose from. Each preset
includes luma primary preset, chroma primary preset, luma secondary
preset, a chroma secondary preset and the number of bits used to signal
the 64x64 level preset. Table 5 summarizes the parameters signaled at the frame level.
At the 64x64 filter block level, the algorithm signals the index for the
specific preset to work with for the 64x64 filter block from among the
set of presets specified at the frame level. Table 6 summarizes the parameters
signaled at the filter block level.
##### Table 5. CDEF parameters signaled at the frame level.
| **Frame level Parameters** | **Values (for 8-bit content)** |
| ----------------------------------------------------------------- | ------------------------------ |
| Luma Damping D | {3, 4, 5, 6} |
| Number of bits used for filter block signaling | {0,..,3} |
| List of 1, 2, 4 or 8 presets. Each preset contains the following: | |
| Luma primary strength | {0,…,15} |
| Chroma primary strength | {0,…,15} |
| Luma secondary strength | {0,1,2,4} |
| Chroma secondary strength | {0,1,2,4} |
##### Table 6. CDEF parameters signaled at the filter block level.
| **Filter-Block-level Parameters** | **Values** |
| ----------------------------------------------------------------- | ------------------------------ |
| Index for the preset to use | Up to 7 |
## Notes
The feature settings that are described in this document were compiled at v0.8.3 of the code and may not reflect the current status of the code. The description in this document represents an example showing how features would interact with the SVT architecture. For the most up-to-date settings, it's recommended to review the section of the code implementing this feature.
## References
\[1\] Steiner Midtskogen and Jean-Marc Valin, The AV1 Constrained
Directional Enhancement Filter (CDEF), 2017.
PK #2]d8[ Appendix-SQ-Weight.mdnu [ # SQ_Weight Appendix
## Description
SQ\_WEIGHT determines if the evaluation of HA, HB, VA, VB, H4 and V4
shapes could be skipped based on the cost of SQ, H and V shapes.
Specifically:
- skip HA, HB and H4 if (valid SQ and H) and (H\_COST \>
(SQ\_WEIGHT \* SQ\_COST) / 100)
- skip VA, VB and V4 if (valid SQ and V) and (V\_COST \>
(SQ\_WEIGHT \* SQ\_COST) / 100)
- `The lower the SQ_WEIGHT, the higher the chance to skip NSQ`
## SQ\_WEIGHT Derivation
SQ\_WEIGHT = **Base** + **Offset**
**Base = f(Preset)**
**Preset(s)** | **BASE**
---------------| ---------
MR | ∞
M0 | 105
M1 | 105
M2 | 100
M3 & beyond | 95
**Offset = f (Target Shape, Block/PIC Type, Coeff Info, QP)**

PS.

## Notes
The feature settings that are described in this document were compiled at v0.8.3 of the code and may not reflect the current status of the code. The description in this document represents an example showing how features would interact with the SVT architecture. For the most up-to-date settings, it's recommended to review the section of the code implementing this feature.
PK #2]jfjp p Appendix-Filter-Intra.mdnu [ # Filter Intra Prediction
## 1. Description of the algorithm
Filter intra mode involves generating intra sample prediction in a
raster scan order based on a set of neighboring predicted samples. As an
illustration of the basic idea consider the sample P shown in Figure 1
below. The intra prediction for sample P is the weighted sum of samples
A, B and C. The latter could be reference pixels for the block and/or
already predicted (i.e. filtered) samples within the same block.

##### Figure 1. Prediction of the sample P using the neighboring samples A, B and C.
The prediction procedure is outlined as follows:
- The predicted block is divided into 4x2 sub-blocks. Example: An 8x8
block is divided into 8 4x2 blocks. See the example of the 8x8 block
in Figure 2 below.

##### Figure 2. Splitting a block into 4x2 blocks.
- For each 4x2 sub-block:
- An array of 7 neighboring samples is prepared. For example, as shown
in Figure 3, the 7 neighboring samples for Block\_0 are reference
samples for the 8x8 block. For Block\_1, the above 5 neighboring
samples are reference sample for the 8x8 block, the left neighboring
samples are filtered samples from Block\_0.

##### Figure 3. Example of reference samples for 4x2 blocks.
- The predicted pixels in the 4x2 sub-block are obtained by filtering
the array of 7 neighboring pixels using the filtering methods. To
illustrate the procedure, consider the 4x2 block shown in Figure 4
below:

##### Figure 4. Reference sample around a 4x2 block.
The predicted  pixels, k=0,…,7, in the 4x2 block is given
by:
av1_filter_intra_taps[mode][k][0] * P0 + av1_filter_intra_taps[mode][k][1] * P1 +\
av1_filter_intra_taps[mode][k][2] * P2 + av1_filter_intra_taps[mode][k][3] * P3 +\
av1_filter_intra_taps[mode][k][4] * P4 + av1_filter_intra_taps[mode][k][5] * P5 +\
av1_filter_intra_taps[mode][k][6] * P6
where mode refers to one of the five supported filter intra modes listed in the table below:
##### Table 1. Filter intra modes.
| **filter\_intra\_mode index** | **Filter intra mode** |
| ----------------------------- | --------------------- |
| 0 | FILTER\_DC\_PRED |
| 1 | FILTER\_V\_PRED |
| 2 | FILTER\_H\_PRED |
| 3 | FILTER\_D157\_PRED |
| 4 | FILTER\_PAETH\_PRED |
The filter coefficients are listed in Table 2 as a function of the intra prediction mode and predicted sample.
##### Table 2. Filter coefficients as a function of the intra prediction mode and the predicted sample.

The best filtering mode (i.e. set of weights) is selected through, for
example, a rate distortion performance measure.
The 4x2 blocks can be processed in a wavefront manner. For the 8x8 block
example shown in Figure 2, the order of processing the 4x2 blocks could
be: Block\_0, then Block\_1 and Block\_2, then Block\_3 and Block\_4, then
Block\_5 and Block\_6.
The Filter Intra feature is applicable only to luma intra prediction and
to blocks that have width and height less than or equal to 32.
## 2. Implementation of the algorithm
##### Control macros/flags
the control flags associated with the filter intra flag are listed in Table 3 below.
| **Flag** | **Level (sequence/picture)** | **Description** |
| ------------------------ | ---------------------------- | -------------------------------------------------- |
| enable\_filter\_intra | sequence | Enable/disable the feature at the sequence header. |
| pic\_filter\_intra\_mode | picture | Enable/disable the feature at the picture level. |
### Main function calls
A diagram of the main function calls associated with the filter intra
algorithm is shown in Figure 5 below. The functions are shown according
to the depth of the function call.

##### Figure 5. Main function calls associated with the filter intra algorithm.
### Candidate Injection
The function ```inject_filter_intra_candidates``` is responsible of
injecting all Filter intra candidates in MD. Candidates are injected for
a given block if the flag ```pic_filter_intra_mode``` is not zero,
```enable_filter_intra``` is set to 1, and both the block width and block
height are smaller than or equal to 32.
A total of 5 intra based candidates are injected where the candidate
field ```filter_intra_mode``` is assigned a value from the following list:
```FILTER_DC_PRED``` / ```FILTER_V_PRED``` / ```FILTER_H_PRED``` /
```FILTER_D157_PRED``` / ```FILTER_PAETH_PRED```. For other regular intra
candidates ```filter_intra_mode``` is assigned a special value
(```FILTER_INTRA_MODES```) to make sure it is not a filter intra candidate.
A special class is assigned to all Filter Intra candidates.
Note that when inter-intra compound is used, no filter intra modes are
allowed in the intra part of the prediction.
### Generation of the filter intra prediction
The generation of the filter predictions is performed in the function
```svt_av1_filter_intra_predictor```. The generation of the prediction is
performed as described above. The block is split into 4x2 blocks and the
predictions for the 4x2 blocks are generated in a raster scan order.
## 3. Optimization of the algorithm
To reduce the complexity associated with the filter intra feature, the latter
is enabled only for base layer pictures when the feature is enabled at the
sequence level (i.e. ```enable_filter_intra``` is set to 1) and ```sc_content_detected```
is zero (i.e. normal video content).
## 4. Signaling
At the sequence header the flag ```enable_filter_intra``` will enable filter
intra in the bit-stream.
When filter intra mode is selected:
- The intra mode that is sent to the decoder is ```DC_PRED```.
- There is no need to send intra angular information in the block info
part of the bit-stream.
- Palette prediction should be disabled.
## Notes
The feature settings that are described in this document were compiled at v0.8.3 of the code and may not reflect the current status of the code. The description in this document represents an example showing how features would interact with the SVT architecture. For the most up-to-date settings, it's recommended to review the section of the code implementing this feature.
PK #2]\+ + 8 Appendix-Compliant-Subpel-Interpolation-Filter-Search.mdnu [ # Sub-pel Interpolation and Interpolation Filter Search
## 1. Description of the algorithm
Motion vectors in SVT-AV1 can be provided with up to eighth-pel accuracy
for luma. The main compliant interpolation filters in AV1 are the
following three 8-tap filters:
- Regular
- Sharp - Sharpens the prediction around the edges.
- Smooth - Blurs the prediction.
Table 1 below lists the different compliant interpolation filters. The
8-tap filters (Regular, Smooth, Sharp) are used with large blocks sizes
(width \> 4). Bilinear filters are also used in fast interpolation
operations (i.e. a speed feature). The 4-tap filters (Regular/Sharp,
Smooth) are used with small block sizes (width \<= 4).
Vertical and horizontal interpolation can be performed using different
1D filters. Interpolation filter selection can be performed at the frame
level or at the block level. Luma filters are different from chroma
filters. As noted in Table 1 below, beyond  position,
filter coefficients are in reverse order as compared to the filter coefficients for the
mirror position before  position.
##### Table 1. Sub-pel interpolation filters.

To illustrate the process by which subpel samples are generated,
consider the diagram shown in Figure 1 below where All sub-pel positions
shown in the diagram are associated with the full-pel position (red
dot). All sub-pel positions are generated directly from full-pel values,
either in one step or in two steps.
- Sub-pel positions in the row of full-pel pixels are generated using
only horizontal filtering. The filter to use is deduced from the
1/16th x offset of the sub-pel position from the full pel position.
- Sub-pel positions in the column of full-pel pixels are generated
using only vertical filtering. The filter to use is deduced from the
1/16th y offset of the sub-pel position from the full pel position.
- Diagonal sub-pel positions are generated in two steps using a
combination of horizontal and vertical filters.
- Horizontal filtering is performed first. The filter to use is
deduced from the  offset of the sub-pel position
from the full-pel position. The buffer for horizontal filtering is
expanded in the y direction on both ends of the block to produce
additional filtered rows that would be used in vertical filtering.
- Vertical filtering is then performed. The filter to use is deduced
from the  offset of the sub-pel position from the
full-pel position.

##### Figure 1. Sub-pel positions.
An illustration of the interpolation process is given in the following
by considering the sub-pel positions illustrated in Figure 2 below.
Consider for example the case of generating half-pel samples around the
best full-pel position (in orange), and assume regular filters are used.
The steps involved in the process are outlined in the following:
- All half-pel positions are generated using the 
filter: **{** **0,** **2,** **-14,** **76,** **76,** **-14,**
**2,** **0** **}**
- Half-pel positions (orange and yellow) in the row of full-pel
pixels:
- Generated using only horizontal filtering.
- The full-pel buffer used in the interpolation is pointed to by
the full-pel position having the same color as the half-pel
position being generated.
- Half-pel positions (green and orange) in the column of full-pel
pixels:
- Generated using only vertical filtering.
- The full-pel buffer used in the interpolation is pointed to by
the full-pel position having the same color as the half-pel
position being generated.
- Diagonal sub-pel positions are generated using a combination of
horizontal and vertical filters.
- Horizontal filtering is performed first. The buffer for
horizontal filtering is expanded in the y direction on both
ends of the block to produce additional filtered rows that
would be used in vertical filtering in the second step (e.g. 3
lines of full-pel pixels are added at the top of the block and
4 lines of full-pel pixels are added at the bottom of the
block).
- Vertical filtering is then performed on the intermediate
results from horizontal filtering.

##### Figure 2. Example of sub-pel calculations.
## 2. Implementation of the algorithm
**Inputs:**
- Reference luminance samples.
- Source luminance samples.
- frame\_type\_neighbor\_array
- interpolation\_type\_neighbor\_array
**Outputs**:
- Predicted block.
**Control macros/flags**:
The control flags associated with interpolation filtering are indicated
in Table 2 below.
Table 2. Control flags for interpolation filtering.
| **Flag** | **Level (sequence/Picture)** | **Description** |
| ---------------------------- | ---------------------------- | --------------------------------------------------------------------------------------------------- |
| allow\_high\_precision\_mv | Picture | When set, it indicates that eighth-pel MV precision is active. |
| interpolation\_search\_level | Picture | Setting to decide on the tradeoff between complexity and performance in interpolation filter search |
## 3. Optimization of the algorithm
To account for the varying characteristics of the video picture in both the horizontal and vertical directions, the selection of the interpolation filter could be done independently for each of the two directions. The selection could be performed through an interpolation filter search, where in addition to the (Regular, Regular) vertical and horizontal filter pair, eight other combination pairs could be evaluated in motion compensation for the same motion vector. The selection of the pair to work with is based on a rate-distortion cost where the filter combination that provides the lowest rate-distortion cost is selected as the best filter pair. The selected filter pair (which corresponds to the best filter combination) is used in the encode pass final motion compensation in the case where the associated candidate is selected as the best candidate in inter-depth decision.
As depicted in the Figure 3 below, the interpolation filter search
consists of three main steps:
- Step1: Test Regular filters for both vertical and
horizontal directions.
- Step2: Fix the horizontal filter to be Regular and
search for the best vertical filter.
- Step3: Fix the vertical filter to be the best filter from
step 2 and search for the best horizontal filter.

##### Figure 3. Diagram illustrating the interpolation filter search process.
The optimization of the interpolation filter search is performed at
two levels. The first level concerns the block sizes where interpolation
search in invoked. The use of interpolation filter search could be
restricted according to block size using the
```interpolation_filter_search_blk_size``` flag as indicated in the
Table 3 and Table 4 below.
##### Table 3. Description of the different interpolation\_filter\_search\_blk\_size settings.
| **interpolation\_filter\_search\_blk\_size** | **Description** |
| -------------------------------------------- | --------------------------------------------------- |
| 0 | Interpolation filter search ON for 8x8 and above. |
| 1 | Interpolation filter search ON for 16x16 and above. |
| 2 | Interpolation filter search ON for 32x32 and above. |
##### Table 4. interpolation\_filter\_search\_blk\_size settings as a function of the encoder preset.
| **Encoder Preset** | **interpolation\_filter\_search\_blk\_size** |
| ------------------ | -------------------------------------------- |
| 0 to 4 | 0 |
| 5 to 8 | 1 |
The second level of optimization concerns where in the MD pipeline
interpolation filter search is used. The flag
```interpolation_search_level``` is used to indicate different levels of
quality-complexity tradeoff points in term of interpolation filter
search, as indicated in the Table 5 below.
##### Table 5. Description of the different settings of the interpolation\_search\_level flag.
| **interpolation\_search\_level** | **Description** |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
| IT\_SEARCH\_OFF (0) | Disable interpolation search |
| IT\_SEARCH\_INTER\_DEPTH (1) | Apply interpolation search at inter depth decision level on the best candidate if it is applicable |
| IT\_SEARCH\_FULL\_LOOP (2) | Apply interpolation search at the full-loop level on the survived mode candidate from the fast-loop when applicable |
| IT\_SEARCH\_FAST\_LOOP\_UV\_BLIND (3) | Apply interpolation in the fast-loop for luminance only when applicable |
| IT\_SEARCH\_FAST\_LOOP (4) | Apply interpolation in the fast-loop when applicable |
The ```interpolation_search_level``` is set as a function the PD\_PASS
as indicated in Table 6 below.
##### Table 6. Description of the interpolation\_search\_level as a function of the encoder preset.

## 4. Signaling
Each of the vertical filter type and horizontal filter type are
signaled independently in the bitstream.
## Notes
The feature settings that are described in this document were compiled at v0.8.3 of the code and may not reflect the current status of the code. The description in this document represents an example showing how features would interact with the SVT architecture. For the most up-to-date settings, it's recommended to review the section of the code implementing this feature.
## References
\[1\] C. Chiang, J Han, S. Vitvitskyy, D. Mukherjee, and Y. Xu,
“Adaptive interpolation filter scheme in AV1,” International Conference on Image Processing, 2017.
PK #2]Ȉd d Appendix-DLF.mdnu [ # Deblocking Loop Filter Appendix
## 1. Description of the algorithm
The deblocking loop filter is used to address blocking artifacts in
reconstructed pictures. The filter was developed based on VP9 deblocking
filter. The filter switches between multiple filters according to the
minimum transform size across the edge and according to whether the
areas around the edge are relatively flat. For a given block, vertical
edges are filtered first, followed by the filtering of the horizontal
edges. The main idea behind the filter can be summarized as follows:
- Pixels on either side of the boundary are not filtered if they show
a clear difference in values, i.e. the edge is most likely a true
edge and not a blockiness artifact.
- If the pixels on both sides of the boundary are to be filtered, the
selection of the filter is performed according to the
guidelines presented in Table 1:
##### Table 1. Summary of the loop filter decisions.
|**Min TX size across the edge**|**Flat areas around the edge**|**Filter (Luma)**|**Filter_Size (Luma)**|**Filter (Chroma)**|**Filter_Size (Chroma)**|
|--- |--- |--- |--- |--- |--- |
|>=TX_8x8|Yes|Filter14 (13-tap)|14|Filter6 (5-tap)|6|
||No|Filter8 (7-tap)/ Filter4|4|Filter4|4|
|TX_8x8|Yes|Filter8 (7-tap)|8|Filter6 (5-tap)|6|
||No|Filter4|4|Filter4|4|
|TX_4x4|-|Filter4|4|Filter4|4|
A block diagram illustrating the flow of the different steps involved in
filtering is given in Figure 1 below.

##### Figure 1. Steps in the loop filter decisions making process.
The steps shown in the block diagram above are now discussed in more
detail.
### Determine the loopfilter level and sharpness
Both the loop filter level and sharpness are frame-level parameters.
- Loop filter level: The loop filter level takes value in \[0, 63\]
and can be set using different methods: (see the functions
```av1_pick_filter_level``` and ```search_filter_level```)
- Set to 0 to disable filtering.
- Set as a function of AC quantization step size.
- Set through the evaluation of filtering using different level values
and selecting the level that yields the lowest distortion in the
filtered frame.
- Sharpness: The sharpness parameter takes value in \[0, 7\] and is an
input parameter to the encoder (--sharpness).
### Filter Strength Parameters: lvl, limit, blimit, thresh
- lvl: At the block level, the loopfilter level is referred to as
lvl. The parameter lvl builds on the loopfilter level for the
frame and includes refinements based on segmentation, coding mode
and reference picture. The lvl parameter is computed as follows:
*lvl = (filter\_level for the frame) + (segment delta) + (mode delta) + (reference delta)*
where
- ```filter_level``` is the loopfilter level for the frame that has
already been determined.
- Segment delta is specified on an 8x8 basis for each of the eight
segments.
- Mode delta and reference delta are determined as follows:
- Define ```scale = 1 << (((filter_level for the frame) + (segment
delta)) >>5 )```
- ```mode delta = mode_deltas * scale```, where ```mode_deltas``` is
obtained from Table 2.
- Reference delta = ```ref_deltas * scale```, where ```ref_deltas``` is
obtained from Table 3 below.
##### Table 2. Mode deltas for the loop filter level.
| **Intra modes** | **mode\_deltas** | **Inter modes** | **mode\_deltas** |
| --------------- | ---------------- | ------------------ | ---------------- |
| DC\_PRED | 0 | NEARESTMV | 1 |
| V\_PRED | 0 | NEARMV | 1 |
| H\_PRED | 0 | GLOBALMV | 0 |
| D45\_PRED | 0 | NEWMV | 1 |
| D135\_PRED | 0 | NEAREST\_NEARESTMV | 1 |
| D113\_PRED | 0 | NEAR\_NEARMV | 1 |
| D157\_PRED | 0 | NEAREST\_NEWMV | 1 |
| D203\_PRED | 0 | NEW\_NEARESTMV | 1 |
| D67\_PRED | 0 | NEAR\_NEWMV | 1 |
| SMOOTH\_PRED | 0 | NEW\_NEARMV | 1 |
| SMOOTH\_V\_PRED | 0 | GLOBAL\_GLOBALMV | 1 |
| SMOOTH\_H\_PRED | 0 | NEW\_NEWMV | 1 |
| PAETH\_PRED | 0 | | |
##### Table 3. Reference deltas for the loop filter level.
| **Reference Picture** | **Default ref\_deltas** |
| --------------------- | ----------------------- |
| INTRA\_FRAME | 1 |
| LAST\_FRAME | 0 |
| LAST2\_FRAME | 0 |
| LAST3\_FRAME | 0 |
| BWDREF\_FRAME | 0 |
| GOLDEN\_FRAME | \-1 |
| ALTREF2\_FRAME | \-1 |
| ALTREF\_FRAME | \-1 |
- shift: The shift parameter is computed as follows:
- If sharpness \> 4, shift = 2.
- Otherwise, if sharpness \> 0, shift = 1.
- Otherwise, shift = 0.
- limit
- If sharpness \> 0, limit = Clip3( 1, 9 - sharpness, lvl \>\>
shift ).
- Otherwise, limit = Max( 1, lvl \>\> shift ).
- blimit = 2 \* (lvl + 2) + limit.
- Thresh = lvl \>\> 4.
### Identify edges to filter
The edges to filter should satisfy the following conditions:
- Transform unit edges AND
- (non-zero level on either side of the edge) AND ((Non-skip inter
blocks on either side of the edge) OR CU edge).
### Filtering Process for luma
The decisions on the filtering operation is based on a number of masks
used to evaluate the level of difference between the samples across the
edge to be filtered. The filtering masks are outlined first, followed by a
description of the filtering decision making process.
*Filter Masks.* The filter masks to be discussed are Filter\_Mask, High
Edge Variance Mask (Hev\_Mask), Flat\_Mask, Flat\_Mask2
- Filter\_Mask: Indicates whether the edge is a true edge or an
artifact, and consequently whether the tested samples are to be
filtered or not. (See the functions ```filter_mask3_chroma``` and
```filter_mask```)
Original idea:
- Let n = 2 when filter\_length = 6 and n = 3 when filter\_length = 8 or 14.
If
- (ABS(  –  ) \> limit, i=1,…,n; OR
- (ABS(  –  ) \> limit, i=1,…,n; OR
- (ABS(  -  ) \* 2 + ABS(  -
 ) / 2 \> blimit)
then the edge is most likely a true edge. In that case, do not filter
the tested samples and set Filter\_Mask to zero. Otherwise,
Filter\_Mask is set to 1 and the tested samples are to be filtered.
- Hev\_Mask: Used to identify edges with large change in pixel values
on either side of the edge. (See the function ```hev_mask```)
If ABS(p1-p0) \> thresh OR ABS(p1-p0) \> thresh, then Hev\_Mask = 1,
else Hev\_Mask = 0.
- Flat\_Mask: Considered when Filter\_Length \>= 6. Indicates whether
samples 0,…,n on each side of the boundary belong to relatively flat
areas, where n = 2 for Filter\_length = 6 and n=3 when
Filter\_length = 8 or 14
Flat\_Mask = 1 when the following conditions are true:
- abs( - ) \<= thresh, i=1,…,n; AND
- abs( - ) \<= thresh, i=1,…,n
Otherwise, Flat\_Mask = 0.
- Flat\_Mask2: Considered when Filter\_Length = 14. Indicates whether
samples 4, 5 and 6 on each side of the boundary belong to relatively
flat areas.
Flat\_Mask2 = 1 when the following conditions are true:
ABS( - ) \<= thresh, i=4,…,6; AND
ABS( - ) \<= thresh, i=4,…,6. Otherwise,
Flat\_Mask2 = 0.
### Filtering decision making process
The steps involved in the filtering operation are as follows:
- Determine min\_ts = The smaller of the two transform sizes on either
side of the edge (e.g. min(TX\_16x16, TX\_8x8) = TX\_8x8)
- Determine filter masks: hevMask, filterMask, flatMask and flatMask2.
- If filterMask == 0, no filtering takes place.
- Otherwise, if ((min\_ts == TX\_4x4) OR (flatMask == 0)), then use
filter4.
- Otherwise, if ((min\_ts == TX\_8x8) OR (flatMask2 == 0)), then use
filter8.
- Otherwise, use filter14.
The filtering decisions are outlined in the diagram shown in Figure 2 below.

##### Figure 2. Flow of the loop filter decision making process.
The different filters that could be considered in the filtering
operation are outlined below. Figure 3 below indicates the positions
of the samples across the horizontal edge to be filtered, with similar
arrangement of the samples for the case of a vertical edge.

##### Figure 3. Sample positions across the horizontal edge to be filtered.
*Filter4:* Modifies up to two samples on each side of the boundary,
depending on High Variance Edge Mask (Hev\_Mask). Rough outline of
the main idea:
- Hev\_Mask = 1  only q0 and p0 are filtered.
- Delta = (( – ) + 3(-))/8
- q0  q0 – Delta; p0  p0 + Delta
- Hev\_Mask = 0  q0, q1, p0 and p1 are filtered.
- Delta = 3(q0-p0)/8
- q0  q0 - Delta; p0  p0 + Delta
- q1  q1 - Delta/2; p1  p1+Delta/2
clamp(x) clamps the value of x to within the interval -128 to 127.
Round2(x,1) returns (x+1)\>\>1.
Implementation (see the function filter4)
ps0 = p0 - 128; ps1 = p1 - 128
qs0 = q0 - 128; qs1 = q1 - 128
filter = clamp( ps1 - qs1 ) if hev_Mask = 1; else 0.
filter = clamp( filter + 3 * (qs0 - ps0) )
filter1 = clamp( filter + 4 ) >> 3
filter2 = clamp( filter + 3 ) >> 3
q0 = clamp( qs0 - filter1 ) + 128
p0 = clamp( ps0 + filter2 ) + 128
if (Hev_Mask == 0)
filter = Round2( filter1, 1 )
q1 = clamp( qs1 - filter ) + 128
p1 = clamp( ps1 + filter ) + 128
*Filter6*
- 5-tap filter: \[1, 2, 2, 2, 1\]
- Applies to chroma planes only.
- Modifies two samples on each side of the edge. (See the function
filter6)
- p1  (p2 \* 3 + p1 \* 2 + p0 \* 2 + q0 + 4)\>\>3;
- p0  (p2 + p1 \* 2 + p0 \* 2 + q0 \* 2 + q1 + 4)\>\>3;
- q0  (p1 + p0 \* 2 + q0 \* 2 + q1 \* 2 + q2 + 4)\>\>3;
- q1  (p0 + q0 \* 2 + q1 \* 2 + q2 \* 3 + 4)\>\>3;
*Filter8*
- 7-tap filter: \[1, 1, 1, 2, 1, 1, 1\]
- Applies to luma plane only.
- Modifies three samples on each side of the edge. (See the function
filter8)
- p2  (p3 + p3 + p3 + 2 \* p2 + p1 + p0 + q0 + 4)\>\>3;
- p1  (p3 + p3 + p2 + 2 \* p1 + p0 + q0 + q1 + 4)\>\>3;
- p0  (p3 + p2 + p1 + 2 \* p0 + q0 + q1 + q2 + 4)\>\>3;
- q0  (p2 + p1 + p0 + 2 \* q0 + q1 + q2 + q3 + 4)\>\>3;
- q1  (p1 + p0 + q0 + 2 \* q1 + q2 + q3 + q3 + 4)\>\>3;
- q2  (p0 + q0 + q1 + 2 \* q2 + q3 + q3 + q3 + 4)\>\>3;
*Filter14*
- 13-tap filter: \[1, 1, 1, 1, 1, 2, 2, 2, 1, 1, 1, 1, 1\]
- Applies to luma plane only.
- Modifies six samples on each side of the edge. (See the function
filter14)
- p5  (p6 \* 7 + p5 \* 2 + p4 \* 2 + p3 + p2 + p1 + p0 + q0 + 8)\>\>4,
- p4  (p6 \* 5 + p5 \* 2 + p4 \* 2 + p3 \* 2 + p2 + p1 + p0 + q0 + q1 +
8)\>\>4
- p3  (p6 \* 4 + p5 + p4 \* 2 + p3 \* 2 + p2 \* 2 + p1 + p0 + q0 + q1 +
q2 + 8)\>\>4
- p2  (p6 \* 3 + p5 + p4 + p3 \* 2 + p2 \* 2 + p1 \* 2 + p0 + q0 + q1 +
q2 + q3 + 8)\>\>4
- p1  (p6 \* 2 + p5 + p4 + p3 + p2 \* 2 + p1 \* 2 + p0 \* 2 + q0 + q1 +
q2 + q3 + q4 + 8)\>\>4
- p0  (p6 + p5 + p4 + p3 + p2 + p1 \* 2 + p0 \* 2 + q0 \* 2 + q1 + q2 +
q3 + q4 + q5 + 8)\>\>4
- q0  (p5 + p4 + p3 + p2 + p1 + p0 \* 2 + q0 \* 2 + q1 \* 2 + q2 + q3 +
q4 + q5 + q6 + 8)\>\>4
- q1  (p4 + p3 + p2 + p1 + p0 + q0 \* 2 + q1 \* 2 + q2 \* 2 + q3 + q4 +
q5 + q6 \* 2 + 8)\>\>4
- q2  (p3 + p2 + p1 + p0 + q0 + q1 \* 2 + q2 \* 2 + q3 \* 2 + q4 + q5 +
q6 \* 3 + 8)\>\>4
- q3  (p2 + p1 + p0 + q0 + q1 + q2 \* 2 + q3 \* 2 + q4 \* 2 + q5 + q6
\* 4 +8)\>\>4
- q4  (p1 + p0 + q0 + q1 + q2 + q3 \* 2 + q4 \* 2 + q5 \* 2 + q6 \* 5 +
8)\>\>4
- q5  (p0 + q0 + q1 + q2 + q3 + q4 \* 2 + q5 \* 2 + q6 \* 7 + 8)\>\>4
## 2. Implementation
**Inputs to dlf\_kernel**: Reconstructed picture from the encode pass.
**Outputs of dlf\_kernel**: Filtered frame, filter parameters.
**Controlling macros/flags**:
##### Table 4. List of loop filter control flags.
| **Flag** | **Level** | **Description** |
| ----------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| loop\_filter\_mode | Picture | Sets the loop filter complexity-performance tradeoff |
| combine\_vert\_horz\_lf | Picture | When set, it implies performing filtering of vertical edges in the current SB followed by filtering of horizontal edges in the preceding SB in the same SB row. When OFF, it implies performing filtering of vertical edges in the current SB followed by filtering of horizontal edges in the same SB. |
The main steps involved in the implementation of the algorithm are
outlined below, followed by more details on some of the important
functions.
The loop filtering operation consists of the following three main steps:
- Initializing the loop filter parameters (limits and thresholds).
- Choosing the optimal loop filter levels
- Applying loop filtering to the frame
The steps above are performed only when (```loop_filter_mode >= 2```).
Otherwise, loop filtering is not applied to the frame. The details of
the three steps mentioned above are outlined in the following.
**Step 1**: Initialize the loop filter limits and thresholds
(```av1_loop_filter_init```)
- Initialize ```lfi->lfthr[lvl].lim = block_inside_limit```, for
0\<=lvl\<=63 (update\_sharpness)
- Initialize ```lfi->lfthr[lvl].mblim = *2 * (lvl + 2) +*
block_inside_limit```, for 0\<=lvl\<=63 (```update_sharpness```)
- Initialize ```lfi->lfthr[lvl].hev_thr = (lvl >> 4)```, for 0\<=lvl\<=63
where block\_inside\_limit is given by
- If sharpness\_lvl \> 0, block\_inside\_lim = Clip3(1, 9 –
sharpness\_lvl, lvl \>\> shift).
- Otherwise, block\_inside\_limit = Max(1, lvl \>\> shift ).
Moreover, ```lf->combine_vert_horz_lf = 1```, implying that both
vertical and horizontal filtering are to be considered.
**Step 2**: Choosing the optimal loop filter levels
(```av1_pick_filter_level```)
In this step, a search is performed for the best loop filter level to
work with. The loop filter levels are:
```filter_level[0]```: Loop filter level for luma vertical edge filtering.
```filter_level[1]```: Loop filter level for luma horizontal edge filtering.
```filter_level_u```: Loop filter level for Cb edge filtering.
```filter_level_v```: Loop filter level for Cr edge filtering.
- if method == LPF\_PICK\_MINIMAL\_LPF
- Set filter\_level\[0\] = filter\_level\[1\] = 0;
- else if method \>= LPF\_PICK\_FROM\_Q
- generate ```filt_guess``` based on the quantization parameter, the encoder
bit depth, the frame type and the picture plane. Update
```filter_level[0], filter_level[1], filter_level_u, filter_level_v```;
- else {
- Get the last frame filter levels ```filter_level[0],
filter_level[1], filter_level_u, filter_level_v```.
- For each of the picture data planes, perform a search for the best
filter level for the picture data plane (search\_filter\_level)
- Set the filter level ```filt_mid``` to the frame level for the last frame,
and the filter search step to 4 if (```fil\_mid < 16```), otherwise it is
set to ```filt_mid/4```.
- Filter the frame with filter level set to filt\_mid and evaluate the
SSE for the filtered frame (```try_filter_frame``` and then
```svt_av1_loop_filter_frame```. See below for more details on the two
functions), update the best SSE ```best_err``` and corresponding filter
level ```filt_best```.
- If (```loop_filter_mode <= 2```),
- **Search Method 1**: filter the frame with filter level set to
(```filt_mid-2```) and evaluate the SSE for the filtered frame
(```try_filter_frame```), update the best SSE ```best_err``` and corresponding
filter level ```filt_best```. Redo the same with filter level set to
(```filt_mid+2```) and update ```best_err``` and ```filt_best```.
- else
- **Search Method 2**: Iterate the search for the best filter level,
starting with filter level (```filt_mid-filter_step```) or
(```filt_mid+filter_step```), depending on the search direction
(```try_filter_frame```). Keep track of the best filtering SSE and filter
level, as well as the search direction. At each iteration, the best
filter level becomes the search starting point for the next iteration.
If the best filter level in the current iteration is the same as in
the previous iteration, halve the ```filter_step```.
- Return the best filter level and corresponding cost.
**Step 3**: Applying loop filtering to the frame based on the selected
loop filter parameters (```svt_av1_loop_filter_frame```).
**More details on** (```try_filter_frame```)
(```try_filter_frame```) is just an intermediate function to prepare for
(```svt_av1_loop_filter_frame```), mainly setting the filter levels,
computing the filtering sse, and resetting the recon buffer. Returns the
filtering SSE.
**More details on** (```svt_av1_loop_filter_frame```)
The function calls that start at ```svt_av1_loop_filter_frame``` are
indicated in Figure 4 below according to the depth of the function
call.

##### Figure 4. Function calls starting at eb\_av1\_loop\_filter\_frame.
The main steps involved in are outlines as follows.
1. (```svt_av1_loop_filter_frame_init```)
- For the given plane, loop over all segments (i.e. segments as defined
by the segmentation feature in AV1 specifications) in the picture
- Loop over the filtering directions (vertical and horizontal)
- Adjust the level calculations for each segment to account for the
level deltas related to each segment, reference pictures, encoding
modes (intra or inter).
2. Loop over all superblocks in the picture and filter each superblock
(```loop_filter_sb```)
- Perform combined filtering of both vertical edges in the current
superblock and filtering of horizontal edges in the *preceding*
superblock in the same superblock row **OR** Perform filtering of all vertical edges in the superblock followed by
filtering all the horizontal edges in the *same* superblock.
(```av1_filter_block_plane_vert```) and
(```av1_filter_block_plane_horz```).
- (``av1_filter_block_plane_vert``) \[The description for
(```av1_filter_block_plane_horz```) is similar, except that the
filtering would be applied to horizontal edges\].
- Loop over rows of 4x4 blocks in the superblock
- For each block that intersects the current row of 4x4 blocks
(```set_lpf_parameters```)
- Determine the transform size to work with for the vertical edges.
(```get_transform_size```)
- For luma plane, If inter block, then ```tx_size =
tx_depth_to_tx_size[0][mbmi->block_mi.sb_type]```, else
```tx_size = tx_depth_to_tx_size[mbmi->tx_depth][mbmi->block_mi.sb_type]```.
Otherwise, ```tx_size``` is determined through the function call
(```av1_get_max_uv_txsize```).
- For luma plane, If inter block and no skip, then ```tx_size =
tx_depth_to_tx_size[mbmi->tx_depth][mbmi->block_mi.sb_type]```
- ```tx_size``` is ultimately set to the width the transform block.
- Determine the loop filter level to use (```get_filter_level```), which
accounts for the loop filter level deltas associated with
segmentation, reference pictures and encoding modes.
- If not at the picture left or top boundaries, filtering is to be
considered if the filter level for the current or the previous 4x4
blocks are non-zero, and \[the current or the previous 4x4 blocks are
inter non-skip blocks, or the edge is a CU edge\]. Under these
conditions, min\_ts, the minimum of the transform sizes associated
with the current and previous 4x4 blocks, is considered. The selection
of the filter length depends on the data plane and min\_ts, as
indicated in the Table above.
- Apply the selected filter to the four samples along the vertical edge.
3. Return the frame filtering sse for the loop filter level and the
picture data plane being considered.
## 3. Optimization of the algorithm
The algorithmic optimization of the loop filter is performed by
considering different loop filter search methods. First, the encoder
mode (```picture_control_set_ptr->enc_mode```) is used to specify the
loop filter mode
(```picture_control_set_ptr->parent_pcs_ptr->loop_filter_mode```)
according to Table 5 below.
##### Table 5. Loop filter mode as a function of the encoder mode.

The ```loop_filter_mode``` is used to specify the filter level search method
in (```av1_pick_filter_level```), either Search Method 1 or Search Method
2. Search Method 2 is more exhaustive than Method 1, and therefore
involves more filtering operations, but could possibly provide better
filtering results. The settings of the filter level search mode as a function
of the ```loop_filter_mode``` are summarized in Table 6.
##### Table 6. Filter Level search Method as a function of the loop\_filter\_mode.
| **loop\_filter\_mode** | **Filter Level Search Method** |
| ---------------------- | ------------------------------ |
| 0 | Loop filter OFF |
| 1 | Loop filter OFF |
| 2 | 1 |
| 3 | 2 |
## 4. Signaling
The loop filter parameters are signaled at the frame level and include
the following parameters: ```filter_level[0]```, ```filter_level[1]```,
```filter_level_u```, ```filter_level_v``` and ```sharpness_level```,
seen in Table 7.
##### Table 7. Frame level loop filter parameters signaled in the bitstream.
| **Parameters** | **Values** |
| ------------------ | ---------- |
| filter\_level\[0\] | {0,…,63} |
| filter\_level\[1\] | {0,…,63} |
| filter\_level\_u | {0,…,63} |
| filter\_level\_v | {0,…,63} |
| sharpness\_level | {0,…,7} |
## Notes
The feature settings that are described in this document were compiled at v0.8.3 of the code and may not reflect the current status of the code. The description in this document represents an example showing how features would interact with the SVT architecture. For the most up-to-date settings, it's recommended to review the section of the code implementing this feature.
## References
\[1\] Zhijun Lei, Srinath Reddy, Victor Cherepanov, and Zhiping Deng,
“GPGPU Implementation of VP9 Inloop Deblocking Filter and Improvements
for AV1 CODEC,” International Conference on Image Processing, pp.
925-929, 2017.
PK #2] svt-av1-decoder-design.mdnu [ # Decoder Design for SVT-AV1 (Scalable Video Technology for AV1)
## Table of Contents
- [Introduction](#introduction)
- [High-level decoder architecture](#high-level-decoder-architecture)
* [Bit-Stream Parse](#bit-stream-parse)
* [Reconstruction](#reconstruction)
* [Loop Filter](#loop-filter)
* [CDEF](#cdef)
* [Loop Restoration](#loop-restoration)
- [Multi-Threaded Architecture](#multi-threaded-architecture)
* [Tile level Parallelism](#tile-level-parallelism)
* [Tile Row-level Parallelism](#tile-row-level-parallelism)
* [Frame Row-level Parallelism](#frame-row-level-parallelism)
* [Job Selection and Sync Points in MT](#job-selection-and-sync-points-in-mt)
- [Frame Level Buffers](#frame-level-buffers)
- [Appendix](#appendix)
* [High-level Data Structures](#high-level-data-structures)
## List of Figures
- [Figure 1](#figure-1): High-Level Decoder Architecture
- [Figure 2](#figure-2): Parse stage Flow
- [Figure 3](#figure-3): Reconstruction stage Flow
- [Figure 4](#figure-4): Loop Filter stage Flow
- [Figure 5](#figure-5): LF Vertical Stage
- [Figure 6](#figure-6): LF Horizontal Stage
- [Figure 7](#figure-7): CDEF stage Flow
- [Figure 8](#figure-8): CDEF Filter Flow
- [Figure 9](#figure-9): Loop Restoration stage Flow
- [Figure 10](#figure-10): Main Thread Flow Chart
- [Figure 11](#figure-11): Worker Thread Flow Chart
- [Figure 12](#figure-12): Tile Parallelism (L > T)
- [Figure 13](#figure-13): Tile Parallelism (L < T)
- [Figure 14](#figure-14): Illustration for Tile Row MT for 4 Tiles and 9 Threads
- [Figure 15](#figure-15): Frame\_i with 5 Threads
- [Figure 16](#figure-16): Sample SB with Blocks numbered (BlockModeInfo)
- [Figure 17](#figure-17): Map with offset for each Block (p\_mi\_offset)
## List of Tables
- [Table 1](#table-1): Important Frame level buffers
## Introduction
This document describes the Intel SVT-AV1 decoder design. In particular, the decoder block diagram and multi-threading aspects are described. Besides, the document contains brief descriptions of the SVT-AV1 decoder modules such as parse, reconstruction, etc. This document is meant to be an accompanying document to the "C Model" source code, which contains the more specific details of the inner working of the decoder.
## High-level decoder architecture
The high-level decoder pipeline is shown in Figure 1. Further details on individual stages are given in subsequent sections. The multi-threading aspect of the decoder is also explained in detail in a separate section. The details of high-level data structures and frame buffers used in the decoder are also covered briefly later.
The major modules of the SVT-AV1 decoder are:
1. Parse
2. Reconstruction
3. Loop filter
4. CDEF
5. Loop Restoration

##### Figure 1: High-Level Decoder Architecture
### Bit-Stream Parse
The Parse stage does all tasks related to OBU reading, arithmetic decoding and related mv prediction (such as find\_warp\_samples, etc.) that produce the necessary mode info and residual data info.
The header level parsing of sequence parameters and frame parameters happens separately via read\_sequence\_header\_obu andread\_frame\_header\_obufunctions. Parse Tile module does the parsing of tile group obu data. Figure 2 shows a typical flow of the parse stage.
**Input** : Bitstream buffer
**Output** : ModeInfo buffer, TransformInfo buffer, Coeff buffer
Picture buffer with predicted values filled for blocks with palette mode
Reconstructed picture buffer without applying post-processing filters for single-thread mode

##### Figure 2: Parse stage Flow
parse\_frame\_tiles() is the function that will trigger the parsing module in SVT-AV1 decoder. Parsing for each tile will be called by start\_parse\_tile(). Then parsing happen for each superblock in a tile by calling the function parse\_super\_block().
Note: The prediction for palette mode happens during the Parse stage itself.
### Reconstruction
This stage performs prediction, coefficient inverse scan, residual inverse quantization, inverse transform and finally generated the reconstructed data without applying post-processing filters. Figure 3 shows a typical flow of the reconstruction stage.
**Input** : ModeInfo buffer, TransformInfo buffer, Coeff buffer
**Output** : Reconstructed picture buffer without applying the post-processing filters.

##### Figure 3: Reconstruction stage Flowdecode\_frame\_tiles() function starts reconstruction at frame level. Then decode\_tile\_job() is called for each tile.
For each Superblock in a tile, decode\_super\_block() function will be called.
The total number of blocks inside a superblock and their corresponding **mode\_info** structure are stored while parsing. This helps to avoid calling of decode\_block() function recursively.
Note: The prediction for palette mode happens during the Parse stage itself.
Note: In single thread mode, decode\_super\_block() will be called immediately after every parse\_super\_block() for better cache efficiency. In this mode decode\_frame\_tiles() will be completely avoided.
### Loop Filter
The loop filter function is to eliminate (or at least reduce) visually objectionable artifacts associated with the semi-independence of the coding of super blocks and their constituent sub-blocks as per section 7.14 of AV1 spec. This stage applies the loop filter for the entire frame. Flow diagram for Loop Filter is below in Figure 4.

##### Figure 4: Loop Filter stage Flow
**Input** : Reconstructed picture buffer
**Output** : Loop Filtered frame.
1. eb\_av1\_loop\_filter\_frame\_init() Initialization of loop filter parameters is performed here.
2. Dec\_loop\_filter\_sb()
1. Apply dec\_av1\_filter\_block\_plane\_vert()
* Loop through each block in SB.
+ Loop through each TUs in a block.
1. Calculate the LF params.
2. Apply LF for each vertical TU edge.
2. Then applydec\_av1\_filter\_block\_plane\_horz().
* Loop through each block in SB.
+ Loop through each TUs in a block.
1. Calculate the LF params.
2. Apply LF for each horizontal TU edge.

##### Figure 5: LF Vertical Stage

##### Figure 6: LF Horizontal Stage
### CDEF
The CDEF performs deringing based on the detected direction of blocks as per section 7.15 of AV1 spec. This stage applies the CDEF for the entire frame. The flow diagram for the CDEF is shown in Figure 7.

##### Figure 7: CDEF stage Flow
**Input:** Output of Loop filtered buffer.
**Output** : CDEF filtered buffer.
Steps involved in CDEF:
1. svt\_cdef\_frame()function will be called to start CDEF for a frame.
2. For each 64x64 superblock function svt\_cdef\_block()will be called.
1. The number of non-skip 8x8 blocks calculated.
2. Store the 3-pixel rows of next SB's in line and column buffer to do CDEF in original pixels.
3. Call eb\_cdef\_filter\_fb() for each 8x8 non-skip block
* Find the direction of each 8x8 block.
* Filter the 8x8 block according to the identified direction- eb\_cdef\_filter\_block\_c().
* Store the results in the destination buffer

##### Figure 8: CDEF Filter Flow
### Loop Restoration
This stage applies the Loop Restoration for the entire frame and the process is defined as per section 7.17 of AV1 spec. The flow diagram for the LR is shown in Figure 9.

##### Figure 9: Loop Restoration stage Flow
**Input** : CDEF filtered reconstructed buffer
**Output** : LR filtered reconstructed buffer
Loop Restoration for a frame starts from the function dec\_av1\_loop\_restoration\_filter\_frame ().
The steps involved are:
1. Call dec\_av1\_loop\_restoration\_filter\_row() for each row of height sb\_size.
* call eb\_dec\_av1\_loop\_restoration\_filter\_unit() for each LR unit of size 64x64
1. Use the stored CDEF/ LF above/below boundaries form neighbor block based on processing row is outer or inner row respectively by calling the function setup\_processing\_stripe\_boundary() .
2. Apply the LR filter (stripe\_filter) based on the type of unit\_lrtype.
3. Restore the LR filtered data back to stripe\_buffer by function restore\_processing\_stripe\_boundary().
## Multi-Threaded Architecture
Parallelism in the decoder could be achieved at multiple levels. Each thread could, for example, be performing a different task in the decoding pipeline. The decoder will use tile level parallelism for tile parsing jobs. Decoder reconstruction jobs will use tile row-level parallelism, whereas all the post-processing filter jobs will use frame row-level parallelism.
Let **N** be the number of threads configured for the decoder. The decoder library created **(N-1)** threads, which are called **worker threads**. The application thread which calls the decode process is called in the **main thread**. Together the decoder will have **N** working threads.
The main thread will perform the following inside the decoder:
1. **Parse** all OBUs completely other than OBU\_TILE\_GROUP
2. Do **MV Projection** Frame Row job if anything is pending
- Wait till all threads have finished the stage
3. **Parse** Tile data if any tile parsing is pending
4. Do **Reconstruct** Tile Row job if anything is pending
5. Do **Loop Filter** Frame Row job if anything is pending
6. Do **CDEF** Frame Row job if anything is pending
- If upscale is enabled, wait till all threads have finished the stage
7. Do **LR** Frame Row job if anything is pending
- Wait till all threads have finished the stage
8. Return the control back to the caller (application)
Figure 10 shows the flow chart of the main thread.
The worker thread will perform the following:
1. Wait for the start frame processing flag
2. Do **MV Projection** Frame Row job if anything is pending
- Wait till all threads have finished the stage
3. **Parse** Tile data if any tile parsing is pending
4. Do **Reconstruct** Tile Row job if anything is pending
5. Do **Loop Filter** Frame Row job if anything is pending
6. Do **CDEF** Frame Row job if anything is pending
- If upscale is enabled, wait till all threads have finished the stage
7. Do **LR** Frame Row job if anything is pending
- Wait till all threads have finished the stage
Figure 11 shows the flow chart of the worker thread.

##### Figure 10: Figure 10 Main Thread Flow Chart

##### Figure 11: Worker Thread Flow Chart
### Tile level Parallelism
The decoder will use tile level parallelism for tile parsing jobs. Let **T** be the number of tiles present in Frame\_i and let **L** be the number of threads working on this frame. Each thread will try to pick up a tile parsing job and execute it as shown in Figure 12 and Figure 13 below.
Please note that the thread number and tile number need not match. Each thread can pick any tile based on job availability. The pictures are just for understanding purpose only.

##### Figure 12: Tile Parallelism (L < T)

##### Figure 13: Tile Parallelism (L > T)
### Tile Row-level Parallelism
Decoder reconstruction uses tile row-level parallelism. Wavefront Processing (WPP) will be used to handle data dependencies. Figure 14 shows 9 threads reconstructing 4 Tiles in Frame\_i with Tile Row-level parallelism. Each thread picks a Tile row MT job and works in a WPP manner.
Each thread will try to pick a unique tile that has not yet processed any row and continues to pick the tile-row jobs from the same tile until no more jobs are present in the same tile. If all the jobs in current tile are picked, it switches to the new tile with maximum number of jobs to be processed. If a unique tile that has not yet processed any row is not found, it picks the tile with maximum number of jobs to be processed.

##### Figure 14: Illustration for Tile Row MT for 4 Tiles and 9 Threads
### Frame Row-level Parallelism
All the post-processing filter jobs will use frame row-level parallelism. Wavefront Processing (WPP) will be used to handle data dependencies if required. LF, CDEF, and LR may work with different unit sizes depending on the available parallelism unit instead of SB.

##### Figure 15: Frame\_i with 5 Threads
Figure 15 shows 5 threads applying post-processing filters on Frame\_i. Each thread picks a Frame row MT job and works in a WPP manner.
### Job Selection and Sync Points in MT
The job selection is controlled using shared memory and mutex. DecMtRowInfo (for Parse Tile, Recon Tile, CDEF Frame, LR Frame), DecMtMotionProjInfo (for Motion Projection), DecMtParseReconTileInfo (for Frame Recon) and DecMtlfFrameInfo (for LF Frame) data structures hold these memory for job selection.
The sync points are controlled using shared memory and mutex. The following are the shared memory used for various syncs inside the decoder stages, like top-right sync.
1. sb\_recon\_row\_parsed: Array to store SB Recon rows in the Tile that have completed the parsing. This will be used for sb decode row start processing. It will be updated after the parsing of each SB row in a tile finished. If the value of this variable is set, the recon of an SB row starts. This check is done before decoding of an SB row in a tile starts inside decode\_tile().
2. sb\_recon\_completed\_in\_row: Array to store SBs completed in every SB row of Recon stage. Used for top-right sync. It will be updated with number of SBs being reconstructed after a recon of SB finished. If recon of 'Top SB' and 'top right SB' is done in the previous row, then only decoding of current SB starts. This check is done before the decoding of SB starts inside the function decode\_tile\_row().
3. sb\_recon\_row\_map: This map is used to store whether the recon of SB row of a tile is finished. Its value is updated after recon of a tile row is done inside decode\_tile() function. If the recon of 'top, top right, current and bottom SB row' is done, then only LF of current row starts. This check is done before starting LF inside the function dec\_av1\_loop\_filter\_frame\_mt().
4. lf\_row\_map: This is an array variable of SB rows to store whether the LF of the current row is done or not. It will be set after the LF of the current row is done. If the LF of the current and next row is done, then only we start CDEF of the current row. This check is done before CDEF of current row starts inside the function svt\_cdef\_frame\_mt().
5. cdef\_completed\_for\_row\_map: Array to store whether CDEF of the current row is done or not. It will be set after the CDEF of the current row is done. If the CDEF of current is done, then only we start LR of the current row. This check is done before LR of current row starts inside the function dec\_av1\_loop\_restoration\_filter\_frame\_mt().
6. Hard-Syncs: The Following are the points where hard syncs, where all threads wait for the completion of the particular stage before going to the next stage, are happening in the decoder.
1. Hard Sync after **MV Projection**. svt\_setup\_motion\_field() is the function where this hard-sync happens.
2. Hard Sync after **CDEF** only when the upscaling flag is present. svt\_cdef\_frame\_mt() is the function where this hard-sync happens.
3. Hard Sync after **LR**. Function where this hard-sync happens is dec\_av1\_loop\_restoration\_filter\_frame\_mt().
## Frame Level Buffers
The following are some important buffers used in the decoder.
| **Structure Description** | **Granularity** |
| --- | --- |
| BlockModeInfo | 4x4 |
| SB info | SB |
| TransformInfo | 4x4 |
| Coeff | 4x4 |
| Delta Q & Delta LF Params | SB |
| cdef\_strength | 64x64 |
| p\_mi\_offset | 4x4 |
Table 1 Important Frame level buffers
### BlockModeInfo
This buffer contains block info required for **Recon**. It is allocated for worst-case every 4x4 block for the entire frame.
Even though the buffer is allocated for every 4x4 in the frame, the structure is not replicated for every 4x4 block. Instead, each block has associated with only one structure even if the block size is more than 4x4. A map with an offset from the start is used for neighbor access purposes. This reduced the need for replication of data structure and better cache efficient usage.
Figure 15 shows a sample superblock split to multiple blocks, numbered from 0 to 18. So 19 BlockModeInfo structures are **continuously populated** from SB start location, corresponding to each block (Instead of replicating the structures for all the 1024 4x4 blocks). Assume this is the first SB in the picture, then Figure 16 shows the map with offset for each location in the SB and stored in p\_mi\_offset buffer. This map will be used for deriving neighbor BlockModeInfo structure at any location if needed.

##### Figure 16: Sample SB with Blocks numbered (BlockModeInfo)

##### Figure 17: Map with offset for each Block (p\_mi\_offset)
### SB info
This buffer stores SB related data. It is allocated for each SB for the entire frame.
### TransformInfo
Transform info of a TU unit is stored in this buffer. It is allocated for each TU unit, the worst case for each 4x4 in a frame.
### Coeff
This buffer contains coeff of each mi\_unit (4x4). Each mi\_unit contains 16 coeffs. For ST, it is allocated for each 4x4 unit for an SB, whereas for MT it is at each 4x4 for the entire frame.
### delta\_q
This buffer is used to store delat\_q params and is allocated at the SB level for the entire frame.
### Delta\_lf
This buffer is allocated at the SB level for the entire frame.
### cdef\_strength
This is allocated at the 64x64 level for the entire frame.
## Appendix
### High-level Data Structures
The following are the high-level data structures in the decoder. Major elements in the structure are explained below.
1. **DecConfiguration :**
* **uint32\_t active\_channel\_count**
* **uint32\_t channel\_id** ID assigned to each channel when multiple instances are running within the same application.
* **uint32\_t compressed\_ten\_bit\_format** Offline packing of the 2bits: requires two bits packed input. Default is 0.
* **EbBool eight\_bit\_output** Outputs 8-bit pictures even if the bitstream has higher bit depth. Ignored if the bitstream is 8-bit. Default is 0.
* **uint64\_t frames\_to\_be\_decoded** Maximum number of frames in the sequence to be decoded. 0 = decodes the full bitstream. Default is 0.
* **EbBitDepth max\_bit\_depth**
* **EbColorFormat max\_color\_format**
* **uint32\_t max\_picture\_height** Picture parameters -height
* **uint32\_t max\_picture\_width** Picture parameters -width
* **uint32\_t num\_p\_frames** Number of frames that can be processed in parallel. Default is 1.
* **int32\_t operating\_point** Default is -1, the highest operating point present in the bitstream. A value higher than the maximum number of operating points present returns the highest available operating point.
* **uint32\_t output\_all\_layers** When set to 1, returns output pictures from all scalable layers present in the bitstream. Default is 0, only one output layer is returned, defined by operating\_point parameter
* **EbBool skip\_film\_grain** Skip film grain synthesis if it is present in the bitstream. Can be used for debugging purpose. Default is 0.
* **uint64\_t skip\_frames** Skip N output frames in the display order. 0 : decodes from the start of the bitstream. Default is 0.
* **uint32\_t stat\_report**
* **uint32\_t threads** Number of threads used by the decoder. Default is 1.
2. **SeqHeader :**
* **EbColorConfig color\_config** Colour Configuration structure
* **DecoderModelInfo decoder\_model\_info** Decoder Mode Information structure
* **uint8\_t decoder\_model\_info\_present\_flag** Specifies whether decoder model information is present in the coded video sequence
* **uint8\_t delta\_frame\_id\_length** Specifies the number of bits used to encode delta\_frame\_id syntax elements
* **uint8\_t enable\_cdef** 1: Specifies that cdef filtering may be enabled. 0: specifies that cdef filtering is disabled
* **uint8\_t enable\_dual\_filter**
+ 1: Indicates that the inter prediction filter type may be specified independently in the horizontal and vertical directions.
+ 0: Indicates only one filter type may be specified, which is then used in both directions.
* **uint8\_t enable\_filter\_intra**
+ 1: Specifies that the use\_filter\_intra syntax element may be present.
+ 0: Specifies that the use\_filter\_intra syntax element will not be present
* **uint8\_t enable\_interintra\_compound**
+ 1: Specifies that the mode info for inter blocks may contain the syntax element interintra.
+ 0: Specifies that the syntax element interintra will not be present
* **uint8\_t enable\_intra\_edge\_filter** Specifies whether the intra edge filtering process should be enabled
* **uint8\_t enable\_masked\_compound**
+ 1: Specifies that the mode info for inter blocks may contain the syntax element compound\_type
+ 0: Specifies that the syntax element compound\_type will not be present
* **uint8\_t enable\_restoration**
+ 1: Specifies that loop restoration filtering may be enabled.
+ 0: Specifies that loop restoration filtering is disabled
* **uint8\_t enable\_superres**
+ 1: Specifies that the use\_superres syntax element will be present in the uncompressed header.
+ 0: Specifies that the use\_superres syntax element will not be present
* **uint8\_t enable\_warped\_motion**
+ 1: Indicates that the allow\_warped\_motion syntax element may be present
+ 0: Indicates that the allow\_warped\_motion syntax element will not be present
* **uint8\_t film\_grain\_params\_present** Specifies whether film grain parameters are present in the coded video sequence
* **uint8\_t frame\_height\_bits** Specifies the number of bits minus 1 used for transmitting the frame height syntax elements
* **uint8\_t frame\_id\_length** Used to calculate the number of bits used to encode the frame\_id syntax element.
* **uint8\_t frame\_id\_numbers\_present\_flag** Specifies whether frame id numbers are present in the coded video sequence
* **uint8\_t frame\_width\_bits** Specifies the number of bits minus 1 used for transmitting the frame width syntax elements
* **uint8\_t initial\_display\_delay\_present\_flag** Specifies whether initial display delay information is present in the coded video sequence.
* **uint16\_t max\_frame\_height** Specifies the maximum frame height minus 1 for the frames represented by this sequence header
* **uint16\_t max\_frame\_width** Specifies the maximum frame width minus 1 for the frames represented by this sequence header
* **EbAv1OperatingPoint operating\_point[MAX\_NUM\_OPERATING\_POINTS]** Operating Point Param structure
* **uint8\_t operating\_points\_cnt\_minus\_1** Indicates the number of operating points minus 1 present in the coded video sequence
* **OrderHintInfo order\_hint\_info** Order Hint Information structure
* **uint8\_t reduced\_still\_picture\_header** Specifies that the syntax elements not needed by a still picture are omitted
* **uint8\_t sb\_mi\_size** Superblock size in 4x4 MI unit
* **BlockSize sb\_size**
* **uint8\_t sb\_size\_log2** Superblock size inlog2 unit
* **uint8\_t seq\_force\_integer\_mv**
+ Equal to SELECT\_INTEGER\_MV indicates that the force\_integer\_mv syntax element will be present in the frame header (providing allow\_screen\_content\_tools is equal to 1).
+ Otherwise, seq\_force\_integer\_mv contains the value for force\_integer\_mv
* **uint8\_t seq\_force\_screen\_content\_tools**
+ Equal to SELECT\_SCREEN\_CONTENT\_TOOLS, indicates that the allow\_screen\_content\_tools syntax element will be present in the frame header.
+ Otherwise, seq\_force\_screen\_content\_tools contains the value for allow\_screen\_content\_tools
* **EbAv1SeqProfile seq\_profile** Specifies the features that can be used in the coded video sequence
* **uint8\_t still\_picture**
+ 1: Specifies that the coded video sequence contains only one coded frame
+ 0: Specifies that the coded video sequence contains one or more coded frames
* **EbTimingInfo timing\_info** Timing Information structure
* **uint8\_t use\_128x128\_superblock**
+ 1: Indicates that superblocks contain 128x128 luma samples
+ 0: Indicates that superblocks contain 64x64 luma samples.
3. **FrameHeader :**
* **uint8\_t all\_lossless** Indicates that the frame is fully lossless at the upscaled resolution
* **uint8\_t allow\_high\_precision\_mv**
+ 0: Specifies that motion vectors are specified to quarter pel precision
+ 1: Specifies that motion vectors are specified to eighth pel precision
* **uint8\_t allow\_intrabc**
+ 1: Indicates that the Intra block copy may be used in this frame.
+ 0: Indicates that the Intra block copy is not allowed in this frame
* **uint8\_t allow\_screen\_content\_tools**
+ 1: Indicates that intra blocks may use palette encoding
+ 0: Indicates that palette encoding is never used
* **uint8\_t allow\_warped\_motion**
+ 1: Indicates that the syntax element motion\_mode may be present
+ 0: Indicates that the syntax element motion\_mode will not be present
* **uint32\_t buffer\_removal\_time[MAX\_NUM\_OPERATING\_POINTS]** Specifies the frame removal time in units of DecCT clock ticks counted from the removal time of the last random access point for operating point op\_num
* **uint8\_t buffer\_removal\_time\_present\_flag**
+ 1: Specifies that buffer\_removal\_time is present.
+ 0: Specifies that buffer\_removal\_time is not present
* **CdefParams cdef\_params** Constrained Directional Enhancement Filter
* **uint8\_t coded\_lossless** Indicates that the frame is fully lossless at the coded resolution of FrameWidth by FrameHeight
* **uint32\_t current\_frame\_id** Specifies the frame id number for the current frame
* **DeltaLfParams delta\_lf\_params** Delta Loop Filter Parameters
* **DeltaQParams delta\_q\_params** Delta Quantization Parameters
* **uint8\_t disable\_cdf\_update** Specifies whether the CDF update in the symbol decoding process should be disabled
* **uint8\_t disable\_frame\_end\_update\_cdf**
+ 1: Indicates that the end of frame CDF update is disabled
+ 0: Indicates that the end of frame CDF update is enabled
* **uint8\_t error\_resilient\_mode**
+ 1: Indicates that error resilient mode is enabled
+ 0: Indicates that error resilient mode is disabled
* **AomFilmGrain film\_grain\_params** Film Grain Parameters
* **uint8\_t force\_integer\_mv**
+ 1: Specifies that motion vectors will always be integers
+ 0: Specifies that motion vectors can contain fractional bits
* **uint32\_t frame\_presentation\_time** Specifies the presentation time of the frame in clock ticks DispCT counted from the removal time of the last random access point for the operating point that is being decoded
* **int32\_t frame\_refs\_short\_signaling**
* **FrameSize frame\_size** Frame Size structure
* **FrameType frame\_type** Specifies the type of the frame
* **InterpFilter interpolation\_filter** Specifies the filter selection used for performing inter prediction
* **uint8\_t is\_motion\_mode\_switchable**
+ 0: Specifies that only the SIMPLE motion mode will be used
* **struct LoopFilter loop\_filter\_params** Loop Filter Parameters
* **uint8\_t lossless\_array[MAX\_SEGMENTS]** Indicates the flag to set coded\_lossless variable
* **LrParams lr\_params[MAX\_MB\_PLANE]** Loop Restoration Parameters
* **uint32\_t mi\_cols**
* **uint32\_t mi\_rows**
* **uint32\_t mi\_stride**
* **uint32\_t order\_hint** Used to compute OrderHint
* **uint32\_t order\_hints[REF\_FRAMES]** Specifies the expected output order for each reference frame
* **uint8\_t primary\_ref\_frame** Specifies which reference frame contains the CDF values and other states that should be loaded at the start of the frame
* **QuantizationParams quantization\_params** Quantization Parameters
* **uint8\_t reduced\_tx\_set**
+ 1: specifies that the frame is restricted to a reduced subset of the full set of transform types
* **uint8\_t ref\_frame\_idx[REF\_FRAMES]** Specifies which reference frames are used by inter frames
* **uint32\_t ref\_frame\_sign\_bias[TOTAL\_REFS\_PER\_FRAME]**
+ 1: Indicates that the end of frame CDF update is disabled
+ 0: Indicates that the end of frame CDF update is enabled
* **uint32\_t ref\_order\_hint[REF\_FRAMES]** Specifies the expected output order hint for each reference frame
* **uint32\_t ref\_valid[REF\_FRAMES]** An array which is indexed by a reference picture slot number
+ 1: Signifies that the corresponding reference picture slot is valid for use as a reference picture
+ 0: Signifies that the corresponding reference picture slot is not valid for use as a reference picture
* **ReferenceMode reference\_mode** Reference Mode structure
* **uint8\_t refresh\_frame\_flags** Specifies the length of the buffer\_removal\_time syntax element
* **SegmentationParams segmentation\_params** Segmentation Parameters
* **uint8\_t show\_existing\_frame**
+ 1: Indicates the frame indexed by frame\_to\_show\_map\_idx is to be output.
+ 0: Indicates that further processing is required
* **uint8\_t show\_frame**
+ 1: Specifies that this frame should be immediately output once decoded
+ 0: Specifies that this frame should not be immediately output
* **uint8\_t showable\_frame**
+ 1: Specifies that the frame may be output using the show\_existing\_frame mechanism
+ 0: Specifies that this frame will not be output using the show\_existing\_frame mechanism
* **SkipModeInfo skip\_mode\_params** Skip Mode Parameters
* **TilesInfo tiles\_info** Tile information
* **TxMode tx\_mode** Specifies how the transform size is determined
* **uint8\_t use\_ref\_frame\_mvs**
+ 1: Specifies that motion vector information from a previous frame can be used when decoding the current frame
+ 0: Specifies that this information will not be used
4. **DecHandle :**
* **struct Av1Common cm**
* **EbDecPicBuf\* cur\_pic\_buf[DEC\_MAX\_NUM\_FRM\_PRLL]**
* **uint32\_t dec\_cnt**
* **EbSvtAv1DecConfiguration dec\_config**
* **EbHandle\* decode\_thread\_handle\_array**
* **FrameHeader frame\_header**
* **uint8\_t is\_lf\_enabled**
* **MainFrameBuf main\_frame\_buf** Main Frame Buffer containing all frame level bufs like ModeInfo for all the frames in parallel
* **int32\_t mem\_init\_done** Flag to signal decoder memory init is done
* **EbMemoryMapEntry\* memory\_map**
* **uint32\_t memory\_map\_index**
* **EbMemoryMapEntry\* memory\_map\_init\_address**
* **EbDecPicBuf\* next\_ref\_frame\_map[REF\_FRAMES]**
* **int32\_t num\_frms\_prll** Num frames in parallel
* **void\* pv\_dec\_mod\_ctxt**
* **void\* pv\_lf\_ctxt**
* **void\* pv\_lr\_ctxt**
* **void\* pv\_main\_parse\_ctxt**
* **void\* pv\_pic\_mgr** Pointer to Picture manager structure
* **EbDecPicBuf\* ref\_frame\_map[REF\_FRAMES]**
* **struct ScaleFactors ref\_scale\_factors[REF\_FRAMES]**
* **int32\_t remapped\_ref\_idx[REF\_FRAMES]**
* **uint8\_t seen\_frame\_header**
* **SeqHeader seq\_header**
* **int32\_t seq\_header\_done** Flag to signal seq\_header done
* **struct ScaleFactors sf\_identity** Scale of the current frame with respect to itself.
* **uint8\_t show\_existing\_frame**
* **uint8\_t show\_frame**
* **uint8\_t showable\_frame**
* **uint32\_t size**
* **EbBool start\_thread\_process**
* **struct DecThreadCtxt\* thread\_ctxt\_pa**
* **EbHandle thread\_semaphore**
* **uint64\_t total\_lib\_memory**
PK #2]XB Appendix-Intra-Block-Copy.mdnu [ # Intra Block Copy
## 1. Description of the algorithm
In intra frames, intra block copy (IBC) allows for the prediction of a
given Intra coded block to be a copy of another intra coded block in the
same frame (i.e. from the reconstructed part of the current frame). The
copied block is specified using a displacement vector (DV). Only integer
DVs are allowed since ```force_integer_mv``` will be equal to 1 for intra
frames. Bilinear interpolation is considered for chroma prediction in
the case of odd DVs. IBC is applicable only to key frames and intra-only
non-key frames. When IBC is active, all in-loop filtering is disabled
for the whole frame, including deblocking filter, CDEF and restoration
filter. The prediction is generated from the reconstructed intra pixels,
where the latter would not have been processed by the in-loop filters
since they are disabled. The predicted block is considered an inter
predicted block using a single reference frame (the current intra
frame), and where the DV is full-pel. Only single-reference mode is
allowed.
A 256 pixels wide area just before the block being predicted is excluded
from the block copy intra search area, i.e. the valid region of the
frame consists of already reconstructed blocks that are 256 pixels away
(in a raster scan order) from the current block. Figure 1 below
illustrates the allowed search area.

##### Figure 1. Diagram showing blocks not allowed in the search area.
The IBC feature is useful in encoding screen content that involves
repeated patterns, text and texture in the same frame.
## 2. Implementation of the algorithm
### Main function calls
A diagram of the main function calls associated with the IBC algorithm
is shown in Figure 2 below. The functions are shown according to the
depth of the function call.

##### Figure 2. Main function calls associated with the IBC algorithm.
### Candidate Injection
In the function ```inject_intra_bc_candidates``` , up to 2 intra candidates
are injected as IBC modes. These modes are considered ```DC_PRED``` modes
when coding the block mode information in the bit stream. Simple
translation is used and no fractional DVs are allowed for this case. For
Chroma, bi-linear interpolation is used to produce predicted pixels. The
two candidates are determined through the ```intra_bc_search``` function
call, which is discussed next.
### DV Search
The function ```intra_bc_search``` performs a search within the current
picture (already decoded reconstructed area). The search is a
combination of a classic Diamond search followed by Hash search (CRC is
used as Hash metric). The search is only performed in full pel
resolution as sub-pel displacements are not allowed in the IBC tool in
AV1.
The decoded reconstructed area is divided into two search areas: Top and
Left. As explained above, due to HW constraints, not all of Top
reconstructed area is used to derive DV vectors. To support wavefront
like SW based processing, more constraints are added to only consider
the valid SBs in such scenario.
More detailed steps involved in the DV search are listed below:
1. Set the reference frame to ```INTRA_FRAME```.
2. Get nearest and near MVs from MV stack for the specified reference
frame. See (```svt_av1_find_best_ref_mvs_from_stack```)
3. Set ```dv_ref``` to either nearest or near.
4. Constrain the ```dv_ref``` mv to be at least a block size away from the
current block, and also to point at least 256 samples away to the
left in the x direction when too close to the tile top boundary.
(```av1_find_ref_dv```)
5. Two types of searches could be performed: Search above the current
block (```IBC_MOTION_ABOVE```) only or search above and to the left of
the current block (```IBC_MOTION_ABOVE``` and ```IBC_MOTION_LEFT```),
depending on the setting of ```ibc_mode```. Up to two dv candidates could
be generated.
6. Limits on mv sizes are computed and refined
(```svt_av1_set_mv_search_range```).
7. Perform full-pel diamond search followed by hash search
(```svt_av1_full_pixel_search```).
8. Make sure returned mv is within the specified mv bounds
(```mv_check_bounds```)
9. Make sure the returned mv meets HW and SW constraints
(```av1_is_dv_valid```)
### Control macros/flags
The Intra block copy algorithm is enabled only when screen content is
detected. Alternatively, there is a manual mode where the user can
configure the encoder to enable or disable IBC via –scm 0/1 in the
encoder command line instruction. With -scm 1, IBC will be used
regardless of the value of the SC detector. Command line option `-intrabc-mode` allows to specify intraBC mode as per table below.
The control flags associated with the IBC feature are listed in Table 1
below.
##### Table 1. Control flags for IBC.
|**Flag**|**Level(Sequence/Picture)**|**Description**|
|--- |--- |--- |
|-scm|Sequence|Command line token. 0: No IBC, 1: IBC ON 2:Auto mode (detector based)|
|-intrabc-mode|Configuration|To specify intraBC mode on command line. 0: OFF, 1-3: IBC ON with complexity as below, -1: Default behavior|
|ibc_mode|Picture|Controls the complexity-quality tradeoff of the feature. 0: OFF, 1: Slow mode, 2: Faster, 3: Fastest|
|allow_intrabc|Picture|For intra pictures and for encoder presets 0 to 5, allow_intrabc setting is the same as the sc_content_detected setting; otherwise, it is set to 0.|
## 3. Optimization of the algorithm
The complexity-quality tradeoffs in the IBC feature are controller
using the ```ibc_mode``` flag. The description of the flag settings is given
in Table 2 below.
##### Table 2. Description of the ibc\_mode settings.
| **ibc\_mode** | **Description** |
| ------------- | --------------------------------------------------------------------------------------------------------------- |
| 0 | OFF |
| 1 | Slow mode. |
| 2 | Fast mode. A more relaxed threshold is used to enable an early exit in the DV search for Nx4 or 4xN blocks. |
| 3 | Faster mode. Only Above search area is considered in the search algorithm |
The settings for ```ibc_mode``` are a function of the encoder preset and
are given in Table 3 below.
##### Table 3. ibc\_mode as a function of the encoder preset.
| **Encoder preset (enc\_mode)** | **ibc\_mode** |
| ------------------------------ | ------------- |
| 0 to 2 | 1 |
| 3 to 8 | 2 |
## 4. Signaling
The main signal that is sent in the bitsream to enable IBC is
```allow_intrabc``` that is sent in the frame header. Note that IBC is only
allowed for Intra coded frames. At the sequence header screen content
tools have to be enabled in order to use IBC at the frame level.
## Notes
The feature settings that are described in this document were compiled at v0.8.3 of the code and may not reflect the current status of the code. The description in this document represents an example showing how features would interact with the SVT architecture. For the most up-to-date settings, it's recommended to review the section of the code implementing this feature.
PK #2]E7D D Appendix-Local-Warped-Motion.mdnu [ # Local Warped Motion Compensation Appendix
## 1. Description of the algorithm
Warped motion modes are inter-prediction modes where the prediction
is generated by applying an (affine) transform to the reference. AV1
has two affine prediction modes: global warped motion and local
warped motion (LW). The latter is discussed in more detail in the following.
AV1 has three types of motion modes that specify the motion of a
block, namely SIMPLE, OBMC and LW. LW motion estimation provides a description
of the type of local motion. Minimal signaling overhead
is realized by signaling one flag in the inter block mode info, and
that only under some conditions. LW cannot be combined with OBMC.
Warped motion compensation concerns the estimation and compensation of small
local motion for a given block. The feature makes use of motion vector information
for neighboring blocks to extract the local motion model parameters. The general
motion model for local warped motion is given by

where  and  represent the sample pixel coordinates in the current
and reference frames, respectively. The decoder performs the same model
estimation, so the encoder needs only to signal whether local warped
motion is the selected mode for the current block and the corresponding
translational model parameters  and
, i.e. the rest of the model parameters are
not signaled in the bitstream.
To simplify the model estimation,  and
 are assumed to represent the entries in the
current block motion vector (**MV** in the Figure 1 below).
Let **MV**=
. Then the above implies . The remaining parameters ,
and  are estimated using a least squares approach.
To illustrate the estimation of the parameters
 and
 using a
least squares approach, consider the example shown in Figure 1 below.

##### Figure 1. Current block in yellow is a 32x32 block. Neighboring blocks that refer to the same reference picture as the current block are in blue. MVs (in orange) for the current block and the blue blocks are used to infer the local warp motion of the yellow block.
In the following, assume the x and y coordinates are specified with
reference to the top left corner of the yellow block. Let
 be the center of the current block, and
 the projection of (C) onto the reference frame using
the motion vector **MV** for the current block. According to the motion
model:

For block 6, define  to be the center of block 6, and
 to be the projection of  onto the reference
frame using the mv  for block 6. Assuming the same
motion model as above, it follows that:

Taking the difference between the two equations above:

The local warp transformation defines how the vector relating  and
 in the source frame is projected into the vector relating  and  in the reference frame.
 =  where 
The vectors  and
 are shown in purple in Figure 1.
The steps above are then repeated for blocks 5 and 3. The least squares
minimization problem is then broken into two estimation problems: One of the estimation problems is to determine
the parameters  and the other is to estimate the parameters
 such that
 and 
where the matrices  and 
are constructed from the data above. The solutions to the least squares
estimation problems are then given as
^{-1}A_1B_1) and ^{-1}A_2B_2)
For implementation purposes, the local warp transform is implemented as
two shears: A horizontal shear and vertical shear. The model matrix H is
then decomposed as follows:

where  are shear
model parameters. The Vertical shear is given by the following model:

whereas the horizontal shear is given by:

The combined transform is given by:

The shear parameters  are determined based on the parameters
 and
. Subpel displacements that results from the application of
the horizontal and vertical shears are evaluated using 8-tap
interpolation filters with  pel precision.
The final warped motion model is applied on an 8x8 basis in the source
frame. The predicted block is constructed by assembling the 8x8
predicted warped blocks from the reference picture.

##### Figure 2. The horizontal, vertical, and combined shears respectively.
At the decoder side, the affine transform parameters are derived at the
block-level using as input the motion vectors of the current and
neighboring blocks.
## 2. Implementation of the algorithm
**Control macros/flags**:
LW can be enabled/disabled at the sequence and the picture level as
indicated in Table 1.
##### Table 1. Control flags/tokens associated with the LW feature.
| **Flag** | **Level (sequence/Picture)** | **Description** |
| ----------------------------- | ---------------------------- | ------------------------------------------ |
| \-local-warp | Config / Sequence | Encoder configuration parameter to enable/disable LW |
| allow\_warped\_motion | Picture | Enable/disable LW |
**Details of the implementation**
As with other prediction mode candidates in the encoder, candidates
for the LW mode are first injected into MD and then processed through
several MD stages of RD optimization. A high-level diagram of the
function calls relevant to the two main LW functions, namely
```inject_inter_candidates``` and ```warped_motion_prediction``` is given in
Figure 3 below.

##### Figure 3. Function calls relevant to the two main LW functions highlighted in blue.
The two main steps involved in the LW processing in MD, namely the
injection of the LW candidates and the generation of the LW predictions
are outlined in the following.
**Step 1: Injection of the LW candidates.**
The injection is performed by the function ```inject_inter_candidates```.
A diagram of the relevant function calls is given in Figure 4.

##### Figure 4. Continuation of Figure 3 with the function calls related to the injection of LW candidates.
1. Check if the current block has overlappable blocks above and/or to
the left of the current block (```has_overlappable_candidates```).
Overlappable blocks are adjacent blocks above or to the left of the
current block that are inter blocks with width \>= 8 and height \>=
8.
2. Inject warped candidate (function
```inject_warped_motion_candidates```) if the current block is such
that width \>= 8 and height \>= 8 and warped\_motion\_injection is
set.
1. Get an MV. The MV would be from List 0 and could correspond
to NEAREST MV, NEAR MV or NEW MV.
2. Compute warped parameters (function
*warped\_motion\_parameters*)
1. **Get warp samples**
1. Get MVs from overlappable neighboring blocks in the
causal neighborhood, i.e. top and left of the current
block. (```wm_find_samples```)
2. Generate the list of warp samples, i.e., selection
of samples (```select_samples```). To perform the
selection of samples, the difference between the MV
for the current block and the MV of the neighboring
block is computed. The sum of the absolute values of
the x and y components of the difference are compared
to a threshold. Neighboring blocks that result in a
large sum are not considered. Stop if number of
samples in the list is small, since the estimated
warp motion parameters would be unreliable.
2. **Warp parameters estimation (function
```svt_find_projection```)**
1. Generate the warp motion parameters with the warp
samples using the least squares fit
(```find_affine_int```). Stop if parameters don’t fit
threshold criteria.
2. Generate warp variables alpha, beta, gamma and delta
for the two shearing operations (i.e., horizontal and
vertical, which combined make the full affine
transformation). (```svt_get_shear_params```). Stop if
the shear parameters are not valid
(```is_affine_shear_allowed```).
3. If not discarded, the LW candidate is added to the RD
andidate list.
**Step 2: Evaluation of the LW candidates in MD**
The generation of the LW predictions in MD is performed using the
function *warped\_motion\_prediction*. A diagram of the associated
function call is shown in Figure 5 below.

##### Figure 5. Continuation of Figure 3 with the function calls related to the evaluation of the LW predictions in MD.
The steps involved in the generation and evaluation of the predictions
are outlined below.
1. Generate warped motion predicted samples for each plane
(```plane_warped_motion_prediction```)
1. plane_warped_motion_prediction: Generates the luma and chroma warped
luma predictions. The chroma predictions are generated for blocks that
are 16x16 or larger.
1. ```av1_dist_wtd_comp_weight_assign```: Returns forward offset and
backward offset for the case of compound reference candidates
and where the inter-inter compound prediction mode is COMPOUND_DISTWTD.
The forward offset and backward offset are used as weights in the generation
of the final prediction.
2. ```av1_make_masked_warp_inter_predictor```: Called only in the case of compound
reference candidate where the inter-inter compound type is COMPOUND_WEDGE or
COMPOUND_DIFFWTD. Generates the predictions for both of those two compound types. The
first step is to build the mask for the case of the COMPOUND_DIFFWTD inter-inter compound
type using the function ```av1_build_compound_diffwtd_mask_d16```. The next step is to generate
the predictions using the function ```build_masked_compound_no_round``` as follows:
1. The function ```av1_get_compound_type_mask``` is called and returns the mask
for either the case of COMPOUND_DIFFWTD or for the case of COMPOUND_WEDGE.
The function ```av1_get_contiguous_soft_mask``` returns the mask for the case of
COMPOUND_WEDGE. For the case of COMPOUND_DIFFWTD, the mask is computed in
the step above.
2. The function ```aom_highbd_blend_a64_d16_mask```/```aom_lowbd_blend_a64_d16_mask```
is the called to perform the blending of the two inter predictions using
the generated mask.
3. ```svt_av1_warp_plane``` is invoked in the case of BIPRED where inter-inter
compound type is COMPOUND_DISTWTD. In this case the ```function highbd_warp_plane```
/ ```warp_plane``` is called and in turn calls the function ```svt_av1_highbd_warp_affine``` /
```svt_av1_warp_affine```. The latter applies the affine transform and generates the warped
motion prediction using the forward offset and backward offset weights associated with the COMPOUND_DISTWTD mode.
This last step is performed at the level of 8x8 blocks, until the prediction for the entire block is generated.
2. ```chroma_plane_warped_motion_prediction_sub8x8```: Generates chroma warped motion
predictions for blocks that are smaller than 16x16. The function
```av1_dist_wtd_comp_weight_assign``` is first called to generate the mask for the
COMPOUND_DISTWTD case. The appropriate function in the function array
```convolve[][][]``` / ```convolveHbd[][][]``` is then called to generate the prediction
using the forward offset and the backward offset weights.
2. Compute RD for the LW prediction. Rate includes the signaling of
the syntax element ```motion_mode```
**Step 3**: Generate the final warped motion predictions in the encode pass.
The main relevant function is warped_motion_prediction which is described above.
## 3. Optimization of the algorithm
The injection of the LW motion candidates is performed if the following is true:
allow_warped_motion is set AND the block has overlappable candidates AND bwidth >= 8
AND bheight >= 8 AND warped_motion_injection is set. The flag warped_motion_injection
is set in signal_derivation_enc_dec_kernel_oq as shown in Table 2 below:
##### Table 2. warped_motion_injection as a function of encoder settings.
| **PD_PASS** | **warped_motion_injection** |
| ------------------------------ | --------------------------------------- |
| PD_PASS_0 | 0 |
| PD_PASS_1 | 1 |
| Otherwise | if sc_content_detected then 0, else 1 |
In mode decision, the picture-level flag allow_warped_motion is set in
signal_derivation_mode_decision_config_kernel_oq as shown in Table 3 below.
##### Table 3. enable_wm setting as a function of encoder preset.

The final setting for allow_warped_motion is determined as follows: allow_warped_motion =
enable_wm AND not KEY_FRAME AND not INTRA_ONLY_FRAME and not error_resilient_mode.
## 4. Signaling
The configuration flag *enable\_local\_warp\_flag* controls the
encoder use of LW at the sequence level. At frame level, the use of
LW is controlled by *allow\_warped\_motion*. At the block level, the
use of LW is signaled by the syntax element *motion\_mode,* which
indicates the type of motion for a block: simple translation, OBMC,
or warped motion.
## Notes
The feature settings that are described in this document were compiled at v0.8.3 of the code and may not reflect the current status of the code. The description in this document represents an example showing how features would interact with the SVT architecture. For the most up-to-date settings, it's recommended to review the section of the code implementing this feature.
## Reference
[1] Sarah Parker, Yue Chen, David Barker, Peter de Rivaz, Debargha Mukherjee,
“Global and Locally Adaptive Warped Motion Compensation in Video Compression,” IEEE International Conference on Image Processing (ICIP), pp. 275-279, 2017.
PK #2]"4 "4 Appendix-TX-Search.mdnu [ # TX Search Appendix
## Transform Search
AV1 provides multiple transform (Tx) type options to work with. Moreover, transform block size could
be the same of smaller than the prediction block size. In the following Tx type search and Tx size search
are discussed in detail.
## I. Tx Type Search
### 1. Description of the algorithm
The AV1 specifications indicate that four transform options could be
considered, namely DCT, Asymmetric Discrete Sine Transform (ADST),
Flipped (reverse) ADST and the Identity transform. A total of 16
transform combinations with independent horizontal & vertical 1D
transform selection are available as shown in Table 1.
##### Table 1. List of the 16 transform combinations available in AV1.
| **Transform Type** | **Vertical** | **Horizontal** |
| ---------------------- | ------------ | -------------- |
| DCT\_DCT | DCT | DCT |
| ADST\_DCT | ADST | DCT |
| DCT\_ADST | DCT | ADST |
| ADST\_ADST | ADST | ADST |
| FLIPADST\_DCT | FLIPADST | DCT |
| DCT\_FLIPADST | DCT | FLIPADST |
| FLIPADST\_FLIPADST | FLIPADST | FLIPADST |
| ADST\_FLIPADST | ADST | FLIPADST |
| FLIPADST\_ADST | FLIPADST | ADST |
| IDTX | IDTX | IDTX |
| V\_DCT | DCT | IDTX |
| H\_DCT | IDTX | DCT |
| V\_ADST | ADST | IDTX |
| H\_ADST | IDTX | ADST |
| V\_FLIPADST | FLIPADST | IDTX |
| H\_FLIPADST | IDTX | FLIPADST |
For best performance, all the applicable transform options would be
evaluated for a given candidate prediction and the transform option that
results in the best cost would be selected. Given that the exhaustive
search option can be computationally expensive, it is desired to find
approaches to evaluate the least number of transform options without
incurrent a significant loss in compression performance. The options
considered in the following are listed below:
- Deciding whether to perform Tx search, and if so where in the
pipeline to perform the search.
- To use a subset of the transform options.
- To exploit already computed cost information for different
prediction candidates for the same block to decide whether to skip
the Tx search for the current candidate based on the cost difference
between the current candidate and the best candidate.
### 2. Implementation of the algorithm
**Inputs**: Prediction candidate.
**Outputs**: Transform type to use.
**Control macros/flags**:
##### Table 2. Tx type search control flags.
| **Flag** | **Level (sequence/Picture)** | **Description** |
| ----------------- | ---------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| tx\_search\_level | Picture | Indicates whether Tx search is to be performed, and if so, whether it would be considered in MD or in the encode pass. |
#### Details of the implementation
The main function calls associated with Tx search in MD and in the
encode pass are outlined in Figure 1 below.

##### Figure 1. Main function calls associated with TX search in MD and in the encode pass.
Tx type search is performed using the functions ```tx_type_Search``` and ```product_full_loop_tx_search```
in MD and the function ```encode_pass_tx_search``` in the encode pass.
### 3. Optimization of the algorithm
A summary of the different optimization approaches considered in Tx
search is presented in Figure 2 below.

##### Figure 2. A Summary of the Tx search optimization approaches.
The optimization of the Tx search is performed using different
approaches as outlined in the following.
**Tx search level**: The Tx search level indicates where in the encoder
pipeline the Tx search would be performed. The three candidate
components of the encoder where Tx search could be performed are the
full-loop in MD, inter-depth decision in MD, and the encode pass.
The flag tx\_search\_level is used to indicate where Tx search
would be performed. Table 3 summarized the values and associated
descriptions of the flag. The settings for tx\_search\_level as a
function of the encoder preset and other settings are given in Table 4.
##### Table 3. tx\_search\_level values and description.
| **tx\_search\_level** | **Value** | **Description** |
| ------------------------ | --------- | ------------------------------------------------------------------- |
| TX\_SEARCH\_OFF | 0 | Tx search OFF |
| TX\_SEARCH\_ENC\_DEC | 1 | Tx search performed only in the encode pass, the lowest complexity option. |
| TX\_SEARCH\_INTER\_DEPTH | 2 | Tx search performed only in inter-depth decision in MD, intermediate complexity level. |
| TX\_SEARCH\_FULL\_LOOP | 3 | Tx search performed only in the full-loop in MD, highest complexity level. |
##### Table 4. tx\_search\_level settings as a function of encoder settings.

**Cost-dependent Tx search**
If Tx search is to be performed in the full-loop in MD, the decision
on whether to perform Tx search could be further refined based on the
difference between the fast loop cost of the current candidate and the
best fast loop cost for the block. If the difference is greater than a
given threshold, Tx search is skipped. The threshold value is specified
by the variable tx\_weight. The values of tx_weight and corresponding
descriptions are given in Table 5. The settings for tx\_weight as a
function of the encoder preset and other settings are given in Table
6.
##### Table 5. tx\_weight values and description.
| **tx\_weight** | **Value** | **Description** |
| ----------------------- | ------------------- | ------------------------------------------ |
| | 0 | Always skip. |
| FC\_SKIP\_TX\_SR\_TH010 | 110 | Skip if difference in cost is 10% or more. |
| FC\_SKIP\_TX\_SR\_TH025 | 125 | Skip if difference in cost is 25% or more. |
| MAX\_MODE\_COST | 13616969489728 \* 8 | No skipping |
##### Table 6. tx\_weight as a function of encoder settings.
|**Preset**|**PD_PASS_0**|**PD_PASS_1**|**PD_PASS_2**|
|--- |--- |--- |--- |
|M0|MAX_MODE_COST|FC_SKIP_TX_SR_TH025|if (tx_search_level == TX_SEARCH_ENC_DEC) then MAX_MODE_COST else FC_SKIP_TX_SR_TH025|
|M1|MAX_MODE_COST|FC_SKIP_TX_SR_TH025|if (tx_search_level == TX_SEARCH_ENC_DEC) then MAX_MODE_COST else FC_SKIP_TX_SR_TH025|
|M2 - M8|MAX_MODE_COST|FC_SKIP_TX_SR_TH025|if is_used_as_reference_flag then FC_SKIP_TX_SR_TH025 else FC_SKIP_TX_SR_TH010|
**Search subset**: If Tx search is performed in either full-loop in MD or
in encode pass in enc/dec, a Tx search subset could be considered
instead of the full Tx search set. The use of a reduced search subset is
specified by the flag ```tx_search_reduced_set```. The values of ```tx_search_reduced_set```
and the corresponding descriptions are given in Table 7. The
settings for ```tx_search_reduced_set``` as a function of the encoder
preset and other settings are given in Table 8.
##### Table 7. tx\_search\_reduced\_set values and description.
| **tx\_search\_reduced\_set** | **Description** |
| ---------------------------- | --------------- |
| 0 | Full Tx set |
| 1 | Reduced Tx set |
| 2 | Two Tx |
##### Table 8. tx\_search\_reduced\_set settings as a function of encoder settings.

## II. Tx Size Search
## 1. Description of the algorithm
For a given block, Tx size search is used to determine the transform
block size that yields the best rate-distortion cost for the block
under consideration. In the current implementation of the Tx size
search feature, only one depth below the current block depth is
considered, as determined by the function ```get_end_tx_depth```. This
is true for inter and intra blocks and for the following block
sizes: 8X8, 8X16, 16X8, 16X16, 16X32, 32X16, 32X32, 32X64, 64X32,
64X64, 4X16, 16X4, 8X32, 32X8, 16X64, 64X16.
## 2. Implementation of the algorithm
**Inputs**: Prediction candidate.
**Outputs**: Transform block size to use.
**Control macros/flags**:
##### Table 9. Control flags associated with Tx size search.
| **Flag** | **Level (sequence/Picture)** | **Description** |
| ---------------------- | ---------------------------- | --------------------------------------------------------------------- |
| md\_atb\_mode | | When set, it allows transform block size search. |
| md\_staging\_skip\_atb | | When set, transform block size search is skipped. |
| use\_intrabc | Block | When set it indicates that Intra Block Copy prediction could be used. |
**Details of the implementation**
The main function calls associated with Tx size search in MD are
outlined in Figure 3 below.

##### Figure 3. Main function calls associated with Tx size search in MD.
Tx size search is enabled currently only in ```MD_stage_2``` since in
```MD_Stage_1``` we have ```md_staging_skip_atb == EB_TRUE```.
The function ```tx_partitioning_path``` performs the Tx size search in MD.
Currently, only the original transform block and the corresponding
one-depth below partitioning transform blocks are evaluated, i.e. only
the original block depth and one depth below are evaluated. The flow of
the evaluation depends on whether the block is an inter coded block or
an intra coded block, as outlined below.
1. In the case of an inter block (i.e. the candidate type is INTER or Intra Block Copy),
the residual block can be computed for the whole
block based on the already computed prediction. This is done in the
function `full_loop_core` through the call to the function
`residual_kernel`.
2. Determine the setting for the flag `tx_search_skip_flag`, which
indicates whether transform type search would be performed or not.
The function `get_skip_tx_search_flag` is used to determine the
setting for the flag.
3. The function `tx_reset_neighbor_arrays` is used to reset the
neighbor arrays.
4. Loop over the depths to be evaluated (i.e. current depth and thenext depth).
a. Initialize the neighbor arrays using `tx_initialize_neighbor_arrays`
b. Loop over the Tx blocks in the depth being evaluated.
1. If the block is not an inter block, then:
* Perform luma intra prediction in av1_intra_luma_prediction.
* Compute the luma resulting residuals in residual_kernel.
2. Perform Tx search for the current Tx block in `tx_type_search`
3. Perform Tx, quantization, inverse quantization, and if spatialSSE, inverse transform. Compute the cost of the current transform type for the transform block size under consideration. All these operations are performed in `product_full_loop`
4. If the block is not an inter block, update both the recon sample neighbor array and the transform-related neighbor array `tx_update_neighbor_array`. Otherwise, update only the transform-related neighbor array in the same function.
c. Estimate the rate associated with signaling the Tx size in `get_tx_size_bits`.
d. Update `best_cost_search` and `best_tx_depth` based on the depths evaluated so far.
## 3. Optimization of the algorithm
The Tx size search optimization is based on checking whether the
parent transform block for the current transform block has all zero
coefficients. If the parent Tx block does not have any non-zero
coefficients, then no further Tx size search is considered. The feature
is controlled by the flag ```tx_size_early_exit```. The flag is used in
```tx_partitioning_path``` to exit the Tx size search if the flag is set. A
description of the flag settings is given in Table 10.
##### Table 10. Description of the tx\_size\_early\_exit settings.
| **tx\_size\_early\_exit** | **Description** |
| ------------------------- | --------------- |
| 0 | Feature OFF |
| 1 | Feature ON |
The flag ```tx_size_early_exit``` is set to 1.
## Notes
The feature settings that are described in this document were compiled at v0.8.3 of the code and may not reflect the current status of the code. The description in this document represents an example showing how features would interact with the SVT architecture. For the most up-to-date settings, it's recommended to review the section of the code implementing this feature.
PK #2]) ) img/film_grain_fig3.pngnu [ PNG
IHDR T gAMA a cHRM z&