python – Best way to replace multiple characters in a string?

ByMartin

python – Best way to replace multiple characters in a string?

Replacing two characters

I timed all the methods in the current answers along with one extra.

With an input string of abc&def#ghi and replacing & -> & and # -> #, the fastest way was to chain together the replacements like this: text.replace(&, &).replace(#, #).

Timings for each function:

  • a) 1000000 loops, best of 3: 1.47 μs per loop
  • b) 1000000 loops, best of 3: 1.51 μs per loop
  • c) 100000 loops, best of 3: 12.3 μs per loop
  • d) 100000 loops, best of 3: 12 μs per loop
  • e) 100000 loops, best of 3: 3.27 μs per loop
  • f) 1000000 loops, best of 3: 0.817 μs per loop
  • g) 100000 loops, best of 3: 3.64 μs per loop
  • h) 1000000 loops, best of 3: 0.927 μs per loop
  • i) 1000000 loops, best of 3: 0.814 μs per loop

Here are the functions:

def a(text):
    chars = &#
    for c in chars:
        text = text.replace(c, \ + c)


def b(text):
    for ch in [&,#]:
        if ch in text:
            text = text.replace(ch,\+ch)


import re
def c(text):
    rx = re.compile(([&#]))
    text = rx.sub(r\1, text)


RX = re.compile(([&#]))
def d(text):
    text = RX.sub(r\1, text)


def mk_esc(esc_chars):
    return lambda s: .join([\ + c if c in esc_chars else c for c in s])
esc = mk_esc(&#)
def e(text):
    esc(text)


def f(text):
    text = text.replace(&, &).replace(#, #)


def g(text):
    replacements = {&: &, #: #}
    text = .join([replacements.get(c, c) for c in text])


def h(text):
    text = text.replace(&, r&)
    text = text.replace(#, r#)


def i(text):
    text = text.replace(&, r&).replace(#, r#)

Timed like this:

python -mtimeit -simport time_functions time_functions.a(abc&def#ghi)
python -mtimeit -simport time_functions time_functions.b(abc&def#ghi)
python -mtimeit -simport time_functions time_functions.c(abc&def#ghi)
python -mtimeit -simport time_functions time_functions.d(abc&def#ghi)
python -mtimeit -simport time_functions time_functions.e(abc&def#ghi)
python -mtimeit -simport time_functions time_functions.f(abc&def#ghi)
python -mtimeit -simport time_functions time_functions.g(abc&def#ghi)
python -mtimeit -simport time_functions time_functions.h(abc&def#ghi)
python -mtimeit -simport time_functions time_functions.i(abc&def#ghi)

Replacing 17 characters

Heres similar code to do the same but with more characters to escape (`*_{}>#+-.!$):

def a(text):
    chars = \`*_{}[]()>#+-.!$
    for c in chars:
        text = text.replace(c, \ + c)


def b(text):
    for ch in [\,`,*,_,{,},[,],(,),>,#,+,-,.,!,$,]:
        if ch in text:
            text = text.replace(ch,\+ch)


import re
def c(text):
    rx = re.compile(([&#]))
    text = rx.sub(r\1, text)


RX = re.compile(([\`*_{}[]()>#+-.!$]))
def d(text):
    text = RX.sub(r\1, text)


def mk_esc(esc_chars):
    return lambda s: .join([\ + c if c in esc_chars else c for c in s])
esc = mk_esc(\`*_{}[]()>#+-.!$)
def e(text):
    esc(text)


def f(text):
    text = text.replace(\, \\).replace(`, `).replace(*, *).replace(_, _).replace({, {).replace(}, }).replace([, [).replace(], ]).replace((, ().replace(), )).replace(>, >).replace(#, #).replace(+, +).replace(-, -).replace(., .).replace(!, !).replace($, $)


def g(text):
    replacements = {
        \: \\,
        `: `,
        *: *,
        _: _,
        {: {,
        }: },
        [: [,
        ]: ],
        (: (,
        ): ),
        >: >,
        #: #,
        +: +,
        -: -,
        .: .,
        !: !,
        $: $,
    }
    text = .join([replacements.get(c, c) for c in text])


def h(text):
    text = text.replace(\, r\)
    text = text.replace(`, r`)
    text = text.replace(*, r*)
    text = text.replace(_, r_)
    text = text.replace({, r{)
    text = text.replace(}, r})
    text = text.replace([, r[)
    text = text.replace(], r])
    text = text.replace((, r()
    text = text.replace(), r))
    text = text.replace(>, r>)
    text = text.replace(#, r#)
    text = text.replace(+, r+)
    text = text.replace(-, r-)
    text = text.replace(., r.)
    text = text.replace(!, r!)
    text = text.replace($, r$)


def i(text):
    text = text.replace(\, r\).replace(`, r`).replace(*, r*).replace(_, r_).replace({, r{).replace(}, r}).replace([, r[).replace(], r]).replace((, r().replace(), r)).replace(>, r>).replace(#, r#).replace(+, r+).replace(-, r-).replace(., r.).replace(!, r!).replace($, r$)

Heres the results for the same input string abc&def#ghi:

  • a) 100000 loops, best of 3: 6.72 μs per loop
  • b) 100000 loops, best of 3: 2.64 μs per loop
  • c) 100000 loops, best of 3: 11.9 μs per loop
  • d) 100000 loops, best of 3: 4.92 μs per loop
  • e) 100000 loops, best of 3: 2.96 μs per loop
  • f) 100000 loops, best of 3: 4.29 μs per loop
  • g) 100000 loops, best of 3: 4.68 μs per loop
  • h) 100000 loops, best of 3: 4.73 μs per loop
  • i) 100000 loops, best of 3: 4.24 μs per loop

And with a longer input string (## *Something* and [another] thing in a longer sentence with {more} things to replace$):

  • a) 100000 loops, best of 3: 7.59 μs per loop
  • b) 100000 loops, best of 3: 6.54 μs per loop
  • c) 100000 loops, best of 3: 16.9 μs per loop
  • d) 100000 loops, best of 3: 7.29 μs per loop
  • e) 100000 loops, best of 3: 12.2 μs per loop
  • f) 100000 loops, best of 3: 5.38 μs per loop
  • g) 10000 loops, best of 3: 21.7 μs per loop
  • h) 100000 loops, best of 3: 5.7 μs per loop
  • i) 100000 loops, best of 3: 5.13 μs per loop

Adding a couple of variants:

def ab(text):
    for ch in [\,`,*,_,{,},[,],(,),>,#,+,-,.,!,$,]:
        text = text.replace(ch,\+ch)


def ba(text):
    chars = \`*_{}[]()>#+-.!$
    for c in chars:
        if c in text:
            text = text.replace(c, \ + c)

With the shorter input:

  • ab) 100000 loops, best of 3: 7.05 μs per loop
  • ba) 100000 loops, best of 3: 2.4 μs per loop

With the longer input:

  • ab) 100000 loops, best of 3: 7.71 μs per loop
  • ba) 100000 loops, best of 3: 6.08 μs per loop

So Im going to use ba for readability and speed.

Addendum

Prompted by haccks in the comments, one difference between ab and ba is the if c in text: check. Lets test them against two more variants:

def ab_with_check(text):
    for ch in [\,`,*,_,{,},[,],(,),>,#,+,-,.,!,$,]:
        if ch in text:
            text = text.replace(ch,\+ch)

def ba_without_check(text):
    chars = \`*_{}[]()>#+-.!$
    for c in chars:
        text = text.replace(c, \ + c)

Times in μs per loop on Python 2.7.14 and 3.6.3, and on a different machine from the earlier set, so cannot be compared directly.

╭────────────╥──────┬───────────────┬──────┬──────────────────╮
│ Py, input  ║  ab  │ ab_with_check │  ba  │ ba_without_check │
╞════════════╬══════╪═══════════════╪══════╪══════════════════╡
│ Py2, short ║ 8.81 │    4.22       │ 3.45 │    8.01          │
│ Py3, short ║ 5.54 │    1.34       │ 1.46 │    5.34          │
├────────────╫──────┼───────────────┼──────┼──────────────────┤
│ Py2, long  ║ 9.3  │    7.15       │ 6.85 │    8.55          │
│ Py3, long  ║ 7.43 │    4.38       │ 4.41 │    7.02          │
└────────────╨──────┴───────────────┴──────┴──────────────────┘

We can conclude that:

  • Those with the check are up to 4x faster than those without the check

  • ab_with_check is slightly in the lead on Python 3, but ba (with check) has a greater lead on Python 2

  • However, the biggest lesson here is Python 3 is up to 3x faster than Python 2! Theres not a huge difference between the slowest on Python 3 and fastest on Python 2!

>>> string=abc&def#ghi
>>> for ch in [&,#]:
...   if ch in string:
...      string=string.replace(ch,\+ch)
...
>>> print string
abc&def#ghi

python – Best way to replace multiple characters in a string?

Here is a python3 method using str.translate and str.maketrans:

s = abc&def#ghi
print(s.translate(str.maketrans({&: &, #: #})))

The printed string is abc&def#ghi.

Leave a Reply

Your email address will not be published. Required fields are marked *