String token values get returned as markup casted back to string

Problem/Motivation

If there is a node field of type plain text, which contains e.g. a & b and is being used as a token, e.g. [node:field_test:value], then the token system returns a & b.

Steps to reproduce

Create a node with a field test and create a node which contains a & b in the test field. Then run drush php and execute this script:

<?php
<?php

use Drupal\node\Entity\Node;

/** @var \Drupal\eca\Token\TokenServices $token */
$token = \Drupal::service('eca.token_services');
$node = Node::load(1); // <= replace the ID with your node ID.

$token->addTokenData('entity', $node);
$result = $token->replace('Hallo [entity:field_test:value] and good bye.');
print(
$result);
?>

The output is a &amp; b.

Proposed resolution

This happens because \Drupal\Core\Utility\Token::doReplace is being called by \Drupal\Core\Utility\Token::replace with $markup = TRUE. That converts the found string value into new HtmlEscapedText($value) which will later be casted back to string during str_replace and that converts all special characters into HTML entities.

If doReplace were called by \Drupal\Core\Utility\Token::replacePlain instead, then no markup would get involved and everything would work as expected.

Two fun facts: nothing in Drupal ever calls replacePlain except a unit test. But if ECA's TokenDecoratorTrait were calling replacePlain instead of replace, everything would be working as expected.

As tokens in ECA are used to grab values to use them elsewhere and not to be rendered, the expected behaviour is to get the same value without that being altered. So, ECA should call replacePlain, I suspect.

Edited by drupalbot
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information