Extension:AbuseFilter/Rules format: Difference between revisions

The rules are a custom language. They are formatted similar to conditionals in a C/Java/Perl-like language.

You can specify a literal by placing it in single or double quotes (for strings), or by typing it in as-is (for numbers, both floating-point and integer). You can get linebreaks with \n, tab characters with \t, and you can also escape the quote character with a backslash.

Use the + (plus) symbol to concatenate two literal strings or the values of two vars with a string value.

Examples

"This is a string"
'This is also a string'
'This string shouldn\'t fail'
"This string\nHas a linebreak"
1234
1.234
-123

You can define custom variables for ease of understanding with the assign symbol := in a line (closed by ;) within a condition. Such variables may use letters, underscores, and numbers (apart from the first character) and are case-insensitive. Example (from w:Special:AbuseFilter/79):

(
	line1:="(\{\{(r|R)eflist|\{\{(r|R)efs|<references\s?/>|</references\s?>)";
	rcount(line1, removed_lines)
) > (
	rcount(line1, added_lines)
)

AbuseFilter has support for non-associative arrays, which can be used like in the following examples.

Caution:

Expressions like page_namespace in [14, 15] may not work as expected. This one will evaluate to true also if page_namespace is 1, 4, or 5. For more information and possible workarounds, please see T181024.

my_array := [ 5, 6, 7, 10 ];
my_array[0] == 5
length(my_array) == 4
int( my_array ) === 4 // Same as length
float( my_array ) === 4.0 // Counts the elements
string(my_array) == "5\n6\n7\n10\n" // Note: the last linebreak could be removed in the future
5 in my_array == true
'5' in my_array == true
'5\n6' in my_array == true // Note: this is due to how arrays are casted to string, i.e. by imploding them with linebreaks
1 in my_array == true // Note: this happens because 'in' casts arguments to strings, so the 1 is caught in '10' and returns true.
my_array[] := 57; // This appends an element at the end of the array
my_array === [ 5, 6, 7, 10, 57 ]
my_array[2] := 42; // And this is for changing an element in the array
my_array === [ 5, 6, 42, 10, 57 ]

You can specify comments using the following syntax:

/* This is a comment */

You can use basic arithmetic symbols to do arithmetic on variables and literals with the following syntax:

• - – Subtract the right-hand operand from the left-hand operand.
• + – Add the right-hand operand to the left-hand operand.
• * – Multiply the left-hand operand by the right-hand operand.
• / – Divide the left-hand operand by the right-hand operand.
• ** – Raise the left-hand operand to the exponential power specified by the right-hand operand.
• % – Return the remainder given when the left-hand operand is divided by the right-hand operand.

The type of the returned result is the same that would be returned by PHP, for which a lot of documentation may be found online. More exhaustive examples may be found in this AF parser test.

You can match if and only if all of a number of conditions are true, one of a number of conditions are true, or one and only one of all conditions are true.

• x | y — OR – returns true if one or more of the conditions is true.
• x & y — AND – returns true if both of the conditions are true.
• x ^ y — XOR – returns true if one, and only one of the two conditions is true.
• !x — NOT – returns true if the condition is not true.

Examples

You can compare variables with other variables and literals with the following syntax:

• <, > – Return true if the left-hand operand is less than/greater than the right-hand operand respectively. Watch out: operands are casted to strings and, like it happens in PHP, null < any number === true and null > any number === false.
• <=, >= – Return true if the left-hand operand is less than or equal to/greater than or equal to the right-hand operand respectively. Watch out: operands are casted to strings and, like it happens in PHP, null <= any number === true and null >= any number === false.
• == (or =), != – Return true if the left-hand operand is equal to/not equal to the right-hand operand respectively.
• ===, !== – Return true if the left-hand operand is equal to/not equal to the right-hand operand AND the left-hand operand is the same/not the same data type to the right-hand operand respectively.

The abuse filter passes various variables by name into the parser. These variables can be accessed by typing their name in, in a place where a literal would work. You can view the variables associated with each request in the abuse log.

Caution:

User-related variables are always available, except for one case: account creation when the creator is not logged in. All variables starting with user_ are affected, except user_type.

Caution:

Always check that the variables you want to use are available for the current action being filtered, e.g. by using the action variable. Failing to do so (for instance using accountname for an edit, or edit_delta for a deletion) will make any code using the variable in question return false.

A variable can be considered protected. For instance, on wikis with temporary accounts enabled, IPs are considered PII and access to them must be restricted. Protected variables and filters that use them are only accessible to maintainers with the abusefilter-access-protected-vars right. Using a protected variable flags the filter as protected as well. The filter subsequently cannot be unprotected, even if it no longer actively uses a protected variable, as its historical logs will remain available.

Logs generated by protected filters can only be viewed by users with the abusefilter-protected-vars-log right.

The default protected variables are defined in AbuseFilterProtectedVariables in extension.json.

When action='move', only the summary, action, timestamp and user_* variables are available. The page_* variables are also available, but the prefix is replaced by moved_from_ and moved_to_, that represent the values of the original article name and the destination one, respectively. For example, moved_from_title and moved_to_title instead of page_title.

Since MediaWiki 1.28 (gerrit:295254), action='upload' is only used when publishing an upload, and not for uploads to stash. A new action='stashupload' is introduced, which is used for all uploads, including uploads to stash. This behaves like action='upload' used to, and only provides file metadata variables (file_*). Variables related to the page edit, including summary, new_wikitext and several others, are now available for action='upload'. For every file upload, filters may be called with action='stashupload' (for uploads to stash), and are always called with action='upload'; they are not called with action='edit'.

Filter authors should use action='stashupload' | action='upload' in filter code when a file can be checked based only on the file contents – for example, to reject low-resolution files – and action='upload' only when the wikitext parts of the edit need to be examined too – for example, to reject files with no description. This allows tools that separate uploading the file and publishing the file (e.g. UploadWizard or Upload dialog ) to inform the user of the failure before they spend the time filling in the upload details.

As noted in the table above, some of these variables can be very slow. While writing filters, remember that the condition limit is not a good metric of how heavy filters are. For instance, variables like *_recent_contributors or *_links always need a DB query to be computed, while *_pst variables will have to perform parsing of the text, which again is a heavy operation; all these variables should be used very, very carefully. For instance, on Italian Wikipedia it's been observed that, with 135 active filters and an average of 450 used conditions, filters execution time was around 500ms, with peaks reaching 15 seconds. Removing the added_links variable from a single filter, and halving the cases when another filter would use added_lines_pst brought the average execution time to 50ms. More specifically:

• Use _links variables when you need high accuracy and checking for "http://..." in other variables (for instance, added_lines) could lead to heavy malfunctioning;
• Use _pst variables when you're really sure that non-PST variables aren't enough. You may also conditionally decide which one to check: if, for instance, you want to examine a signature, check first if added_lines contains ~~~;
• In general, when dealing with these variables, it's always much better to consume further conditions but avoid computing heavy stuff. In order to achieve this, always put heavy variables as last conditions.

Last but not least, note that whenever a variable is computed for a given filter, it'll be saved and any other filter will immediately retrieve it. This means that one single filter computing this variable counts more or less as dozens of filters using it.

The following special keywords are included for often-used functionality:

• like (or matches) returns true if the left-hand operand matches the glob pattern in the right-hand operand.
• in returns true if the right-hand operand (a string) contains the left-hand operand. Note: empty strings are not contained in, nor contain, any other string (not even the empty string itself).
• contains works like in, but with the left and right-hand operands switched. Note: empty strings are not contained in, nor contain, any other string (not even the empty string itself).
• rlike (or regex) and irlike return true if the left-hand operand matches (contains) the regex pattern in the right-hand operand (irlike is case insensitive).
  • The system uses PCRE.
  • The only PCRE option enabled is PCRE_UTF8 (modifier u in PHP); for irlike both PCRE_CASELESS and PCRE_UTF8 are enabled (modifier iu).
• if ... then ... end
• if ... then ... else ... end
• ... ? ... : ...
• true, false, null

Examples

A number of built-in functions are included to ease some common issues. They are executed in the general format functionName( arg1, arg2, arg3 ), and can be used in place of any literal or variable. Its arguments can be given as literals, variables, or even other functions.

Operations are generally done left-to-right, but there is an order to which they are resolved. As soon as the filter fails one of the conditions, it will stop checking the rest of them (due to short-circuit evaluation) and move on to the next filter. The evaluation order is:

1. Anything surrounded by parentheses (( and )) is evaluated as a single unit.
2. Turning variables/literals into their respective data. (e.g., page_namespace to 0)
3. Function calls (norm, lcase, etc.)
4. Unary + and - (defining positive or negative value, e.g. -1234, +1234)
5. Keywords (in, rlike, etc.)
6. Boolean inversion (!x)
7. Exponentiation (2**3 → 8)
8. Multiplication-related (multiplication, division, modulo)
9. Addition and subtraction (3-2 → 1)
10. Comparisons (<, >, ==)
11. Boolean operations (&, |, ^)
12. Ternary operator (... ? ... : ...)
13. Assignments (:=)

• A & B | C is equivalent to (A & B) | C, not to A & (B | C). In particular, both false & true | true and false & false | true evaluates to true.
• A | B & C is equivalent to (A | B) & C, not to A | (B & C). In particular, both true | true & false and true | false & false evaluates to false.
• added_lines rlike "foo" + "|bar" is wrong, use added_lines rlike ("foo" + "|bar") instead.

The condition limit is (more or less) tracking the number of comparison operators + number of function calls entered.

Further explanation on how to reduce conditions used can be found at Extension:AbuseFilter/Conditions .

Although the AbuseFilter examine function will identify "rollback" actions as edits, the AbuseFilter will not evaluate rollback actions for matching.^[11]

• PCRE pattern syntax
• Edit filters of cross-wiki interest
• Extension:AbuseFilter/Conditions