Adding custom information on pandas progress_apply

[AliAbdelaal]

• I have marked all applicable categories:
  • exception-raising bug
  • visual output bug
  • documentation request (i.e. "X is missing from the documentation." If instead I want to ask "how to use X?" I understand [StackOverflow#tqdm] is more appropriate)
  • new feature request
• I have visited the [source website], and in particular
  read the [known issues]
• I have searched through the [issue tracker] for duplicates
• I have mentioned version numbers, operating system and
  environment, where applicable:

  import tqdm, sys
  print(tqdm.__version__, sys.version, sys.platform)

4.42.1 3.7.6 (default, Jan 8 2020, 19:59:22)
[GCC 7.3.0] linux

[source website]: https://github.com/tqdm/tqdm/
[known issues]: https://github.com/tqdm/tqdm/#faq-and-known-issues
[issue tracker]: https://github.com/tqdm/tqdm/issues?q=
[StackOverflow#tqdm]: https://stackoverflow.com/questions/tagged/tqdm

Can't add a custom information message such as `set_postfix` for example when dealing with pandas dataframe `progress_apply` function.

[casperdcl]

#from tqdm.auto import tqdm
from tqdm import tqdm
import pandas as pd

df = pd.DataFrame(...)
tqdm.pandas(desc="customised", postfix={"hello": -1})
df.progress_apply(...)

tqdm.pandas(desc="remove postfix", postfix=None)
df.progress_apply(...)

I agree this is clunky but (kw)arguments to df.progress_apply are all passed to df.apply

[AliAbdelaal]

#from tqdm.auto import tqdm
from tqdm import tqdm
import pandas as pd

df = pd.DataFrame(...)
tqdm.pandas(desc="customised", postfix={"hello": -1})
df.progress_apply(...)

tqdm.pandas(desc="remove postfix", postfix=None)
df.progress_apply(...)

I agree this is clunky but (kw)arguments to df.progress_apply are all passed to df.apply

Can I change the postfix string based on the iteration dynamically instead of having some fixed postfix ?

[casperdcl]

That's not exactly currently supported, and it wouldn't be easy to add without breaking backward compatibility.

The easiest approach would be explicitly defining a bar_format, e.g:

tqdm.pandas(
	bar_format="{l_bar}{bar}| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, {rate_fmt}, iter={n}]")
df.progress_apply(...)

[AliAbdelaal]

Cool, thank you so much for your help.

[armsp]

Currently I use it as follows -

from tqdm import tqdm
pbar = tqdm()

# inside of a class's __init__
        pbar.pandas(bar_format="{l_bar}{bar}| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, {rate_fmt}]")
# inside the function that progress_apply calls -
        pbar.set_postfix({"Accuracy":self.accuracy})  # Update with dynamic info

# progress_apply
        df.progress_apply(self.prediction, axis=1)) 
# in the end
pbar.close()

The result is usable, but could definitely be made prettier -

0it [00:00, ?it/s]
0it [04:24, ?it/s, Accuracy=0.559]
 16%|███████████████████████████████▋                                                                 | 119/743 [03:34<20:11,  1.94s/it]                                                                                     

Not sure if that's the best approach, how do you use it @casperdcl ? Because the ideal solution would be having that custom postfix displayed on the right and not the top.