[Bug] Plotly Express does not respect trace-specific color sequences defined in `template`

[antonymilne]

Hello! I'm a huge fan of plotly and especially plotly.express, but there's something that's bothers me a lot...

Problem

There's no way to make a template use different color sequences for different trace types in plotly express. Currently it always uses template.layout.colorway (unless explicitly overridden) and ignores per-trace defaults in template.data.<trace_type>.

Example

I'll use histogram and scatter here as an example but it applies very generally.

Let's say I want to use colors ["orange", "purple", "grey"] for a histogram. These are different from the default qualitative palette template.layout.colorway. Plotly enables you to give trace-specific colors like the below but it doesn't work: Plotly express ignores template.data.histogram and uses the same template.layout.colorway palette for both plots.

import plotly.express as px
import plotly.io as pio
import plotly.graph_objects as go

histogram_colors = ["orange", "purple", "grey"]

# Set histogram-specific colors in the template
histogram_template = go.layout.Template(
    data={"histogram": [{"marker": {"color": histogram_colors}}]}
)
template = pio.templates.merge_templates("plotly", histogram_template)

df = px.data.iris()

# Expectation: scatter uses global layout.colorway; histogram uses ["orange","purple","grey"]
# Actual behaviour: both use global layout.colorway
fig1 = px.scatter(df, x="sepal_length", y="petal_length", color="species", template=template)
fig2 = px.histogram(df, color="species", template=template)

Current workarounds

• supply color_discrete_sequence or color_discrete_map to each px call
• use multiple templates with different layout.colorway and apply a different template to each chart

Why this matters

In reality I don't actually want to completely change colors between histogram and scatter. Actually what I'd like to do is apply the same palette but desaturated to the histogram. Seaborn supports this e.g. barplot(saturation=0.75).

I would like to be able to achieve this with a single template and without manually supplying arguments on per-figure basis.

Proposed solution

Extend plotly express's discrete color resolution to look at trace-specific colors in preference to template.layout.colorway. A single Plotly template already has space to supply a different color palette on each trace by using template.data, and Plotly express does look in template.data but only for a very limited number of things, e.g. it looks at template.data.scatter for line_dash_sequence. But color_discrete_sequence is always taken from template.layout.colorway rather than looking at template.data.

Instead, plotly could take color_discrete_sequence from the relevant template.data trace and then, if that's not defined, fall back on the current mechanism to default to template.layout.colorway.

Alternative solutions

• a new saturation argument in plotly schema, like there's already opacity. This wouldn't allow arbitrary changes between different trace types but just to desaturate
• a new saturation argument in plotly.express that would automatically apply the color transformation, like in seaborn. No need to modify the plotly figure schema
• ability to define different color sequences for different trace types in plotly express. plotly.express.defaults currently defines a single color_discrete_sequence, but there could be a new color_discrete_sequence_by_trace = {"histogram": ..., ...} or similar

[antonymilne]

@huong-li-nguyen pointed me to this comment originally posted by @nicolaskruchten in plotly_express issue #70:

In a template, we have color scales and sequences in a few places in layout such as layout.colorway, layout.piecolorway, layout.sunburstcolorway, layout.colorscale.sequential, layout.colorscale.sequentialminus, layout.colorscale.diverging but also on a per-trace-type basis: a template can specify one colorscale for scatter and another for heatmaps, and it can also specify that the first scatter is one color and the second is another, at which point I think that scatter traces will end up alternating between the two colors. So... yeah, there are a lot of ways to specify colors in a template.

All that said, px could just ignore most of that complexity and try to read from template.layout.colorway and template.layout.colorscale.sequential and call it a day :)

So it would seem that there are already places in layout to specify multiple colorways for five trace types (pie, funnelarea, sunburst, icicle, treemap). However, these do not exist for most trace types and are ignored by plotly express anyway.

In short I would like to ask whether the decision to "ignore most of that complexity" could be reconsidered to allow per-trace-type color scales. Or for there to be some other easy way to tweak a single color palette (e.g. by its saturation).

[emilykl]

@antonymilne Thanks for the suggestion!

I agree that having a way to specify different default colorways for different trace types would be a useful feature.

There is some complexity here with adding colorway into the template under data, though. The template follows a very specific format which mirrors the available Plotly.js attributes for layout and traces. Attributes passed under data are applied directly to the corresponding traces. Currently, Plotly.js traces have no colorway attribute, which is why passing colorway it in the template under data has no effect. The reason traces don't have a colorway is because individual Plotly.js traces use either a single solid color, or a continuous colorscale based on the data. Plots with discrete color scales in Plotly Express are represented as multiple single-color traces in Plotly.js. The colorway attribute is therefore a property of the entire plot, which tells Plotly.js which color to use each time a new trace is added.

Off the top of my head I can think of a few possible ways of implementing this feature:

• In Plotly.py: Modify px.defaults to support per-trace-type color_discrete_sequence values. I'm not immediately sure what the best API would be; perhaps we could allow color_discrete_sequence to be either a color sequence, or a dictionary mapping trace types to color sequences.

• In Plotly.js: Add additional <tracetype>colorway layout attributes for every trace type which uses colorways. I don't love this option as it would add a lot of top-level attributes to the layout, but it would provide the requested functionality. These <tracename>colorway attributes could then be given values in the template.

• In Plotly.js: Add a new layout attribute, something like tracecolorways, which accepts a dictionary mapping trace names to colorways. Solves the problem of adding lots of top-level attributes to layout but would have to think some more about the pros and cons of this approach. Also would be redundant with existing <tracetype>colorway.

Open to other suggestions and we're happy to discuss further if you'd like to make an API proposal and/or work on a PR.

[antonymilne]

Thanks very much for the comment @emilykl! In principle I'd love to make an API proposal and/or work on a PR, but let me just try and understand a few things better first!

There is some complexity here with adding colorway into the template under data, though. The template follows a very specific format which mirrors the available Plotly.js attributes for layout and traces. Attributes passed under data are applied directly to the corresponding traces. Currently, Plotly.js traces have no colorway attribute, which is why passing colorway it in the template under data has no effect. The reason traces don't have a colorway is because individual Plotly.js traces use either a single solid color, or a continuous colorscale based on the data. Plots with discrete color scales in Plotly Express are represented as multiple single-color traces in Plotly.js. The colorway attribute is therefore a property of the entire plot, which tells Plotly.js which color to use each time a new trace is added.

So I understand why what I wrote above would never work (data={"histogram": [{"marker": {"color": histogram_colors}}]}), but I believe that this would be valid:

histogram_template = go.layout.Template(
    data=[{"histogram": [{"marker": {"color": color}}]} for color in histogram_colors]
)

It won't do anything currently, but doesn't this give a potential space to store trace-specific colors in data? As per the documentation, "They should be objects with format: {layout: layoutTemplate, data: {[type]: [traceTemplate, ...]}, ...} layoutTemplate and traceTemplate are objects matching the attribute structure of layout and a data trace. Trace templates are applied cyclically to traces of each type".

I guess the problem with doing it this way will be that you need to create color_discrete_sequence in a similar way to how the symbol_sequence is created:

plotly.py/plotly/express/_core.py

Lines 1050 to 1052 in 11d50b0

	args["symbol_sequence"] = [
	scatter.marker.symbol for scatter in args["template"].data.scatter
	]

The above is hardcoded to look at data.scatter, whereas to get trace-specific logic you'd have to do something like this:

args["color_discrete_sequence"] = [<trace_type>.marker.color for color in args["template"].data.<trace_type>]

and maybe <trace_type> isn't available at this point?

In fact, now that I think about it again, maybe the issue (and potential solution?) here might be more general here than just trace-specific colors. For example, you can set a template with data={"pie": {"hole": 0.5}} which is respected by plotly in general but ignored by plotly express.

I wonder whether there's some more general mechanism to try and apply template.data to plotly express charts, or whether we just need to deal with these on a case by case basis i.e. one solution for color_discrete_sequence and another for the pie chart hole?

---

Assuming these need to be solved separately, I just wanted to understand a bit more context for the solutions you proposed:

1. You didn't comment on the possibility of a saturation argument which was another possible solution (either in plotly.js or just in plotly express). Is this because you don't like it? (I'm not particular attached to it, but just wanted to understand.) In my mind, this might solve ~95% cases (completely made up statistic) that require trace-specific colors but with a much simpler user API that potentially matches seaborn. The con is that it's not such a general solution for the ~5% of cases where people really want to encode completely different color schemes for different trace types encoded in a template rather than just altering saturation.
2. Do you know why there are five trace types (pie, funnelarea, sunburst, icicle, treemap) that already have layout.<trace_type>colorway attributes? Is this just historical accident or a deliberate choice of trace types?

[emilykl]

So I understand why what I wrote above would never work (data={"histogram": [{"marker": {"color": histogram_colors}}]}), but I believe that this would be valid:

histogram_template = go.layout.Template(
    data=[{"histogram": [{"marker": {"color": color}}]} for color in histogram_colors]
)

It won't do anything currently, but doesn't this give a potential space to store trace-specific colors in data?

Yes actually, I hadn't even looked at it that way, but I think you're exactly right. With a small tweak to the syntax, I think that code should be expected to apply a custom colorscale to the histogram traces only:

histogram_template = go.layout.Template(
    data={
        "histogram": [{"marker": {"color": color}} for color in colors]
    }
)

However, you're also correct that these colors are ignored by Plotly Express! The template does appear in the plot JSON, but it seems Plotly Express is hard-coding colors on trace creation, and thus the template colors are ignored.

So I would reframe this issue as "Plotly Express ignores marker colors in template" and consider it a bug, which could be fixed by modifying the trace-creation logic in Plotly Express.

In fact, now that I think about it again, maybe the issue (and potential solution?) here might be more general here than just trace-specific colors. For example, you can set a template with data={"pie": {"hole": 0.5}} which is respected by plotly in general but ignored by plotly express.

I think you're correct that this issue may be broader than just marker colors, but FWIW, I see the expected hole size of 0.5 when running the code below. So some of the template values in data are being applied correctly. (In this case I think that the default None value for hole results in no value being hard-coded into the traces, which causes Plotly.js to fall back to the hole value provided in the template. Marker colors, on the other hand, have some special logic applied when splitting the data passed to Plotly.py into separate traces, so that trace-splitting logic would have to be modified to not hard-code individual colors when marker.color has a value in the template for that trace type.)

import plotly.graph_objects as go
import plotly.express as px
import plotly.io as pio

pie_template = go.layout.Template(
    data={"pie": [{"hole": 0.01}]}
)
template = pio.templates.merge_templates("plotly", pie_template)

fig = px.pie(values=[1,2,3], names=["A", "B", "C"], template=template)
fig.show()

---

In summary, unless I'm missing something, I think this feature could be achieved by modifying Plotly Express to respect template colors when creating traces.

In response to your final two questions:

• Implementing saturation is another possibility, although it might be a moot point if the above solution works
• I'm not sure of the historical reasons for the existence of the layout.<trace_type>colorway attributes, I would have to do some research

[antonymilne]

Hi @emilykl, thanks for all this! I've raised a PR #5437 that implements the trace-specific color sequences in plotly express. As you suggested, I treated it as a bug fix rather than new feature. Please do take a look and see what you think.

You're absolutely right that pie chart hole is working correctly in plotly express. I'm not sure why it didn't work for me before 🤔 So no problem there.

[emilykl]

Just reviewed — great PR! Thanks @antonymilne !