Xmart OS

The next generation in-car OS for a new sedan, P7, that is intelligent, inclusive, and safe.

Interaction Design
Usability Testing

Launch Review

Motion Design

Documentation Guidelines

Launch Review

Motion Design

Documentation Guidelines

Tsahao Yu - HMI Design Intern (Me)
Shuai Deng - Senior HMI Designer
Yingjie Deng - Visual Designer
Strict production deadline
Hardware ergonomics limitation

May - Aug 2019

Launched in Jul 2020

MY contribution

On-screen interfaces, usability testing, and design documentation guidelines

At Xpeng, my work is in three parts. One is HMI (Human-Machine Interface) design of System UI of the new Xmart OS, under close collaboration with PMs, visual designers, and engineers. I designed the dock bar, status bar, and proposed an innovative smart suggestion feature. Designs were shipped in Jul 2020.

As an intern, I also assisted other designers to conduct launch reviews, where I spotted and logged issues, and communicated with engineers and designers to find solutions.

Additionally, I also created a documentation guideline, which was used by the design team ever since. The guideline helped designers to standardize the format to make various UI statuses easier to understand.

business goal

Deliver a more intelligent mobility experience

Xpeng at that time was an EV (electric vehicle) startup with only one launched entry-level product. The company decided to develop a premium product with top-notch intelligent ability. Therefore, Xpeng can create a key differentiator in the EV industry, potentially leading to better sales and better brand awareness.


Enabled by modern technologies, mobility experience becomes more human-centered.

I conducted secondary research to understand the historical development of HMI (Human-Machine Interface). While traditionally mechanical performance is the focus of car design, I identified that human experience becomes the new focus of the design for mobility. This is enabled by the progression of technologies like the ability to sense the environment and understand the people. I identified design goals as follows and brought them into designing System UI.

Consider passengers’ experience

Traditionally car controls are designed in a driver-centric way. The dashboard looks confusing and almost intimidating to the passenger.

Leverage connectivity to provide context-aware experience

Traditional car functions are based on merely hardware. Modern EVs have the connectivity to internet, and more available data from sensors, which can be leveraged to provide context-aware experience beyond hardware.

Balance functionality with safety

Greater functionality increases the complexity, extending the time a driver needs to interact with the car and negatively affecting driving safety. Therefore we should design a safer experience.
Design goals identified from company vision and secondary research
initial layout

Legacy design in a new display ratio as the starting point

Carrying over the UI we had from previous generation and fitting it into the landscape display ratio, I had this UI layout as a foundation to build upon.
the starting point of xmart os 2.0
Adaptive AC that considers passengers’ experience
Research team identified that key passenger tasks are climate control and music. Initially I moved the temperature adjustments to the side of the screen for easier access for both drivers and passengers.

initially I moved the temperature control from the center to the sides

However, I realized I went with a wrong direction because the true pain point is passenger not having the agency to control AC. This is because drivers often turn SYNC mode on, which means the temperature on the passenger side will be controlled together by the driver. This gives an unwelcoming feeling to the passenger.

I reapproached the problem and designed an adaptive AC control by leveraging the seat occupancy sensor so that the car is contextually aware. Driver owns the control when there is no passenger. But the system will provide passenger the agency to control AC the moment they step into the car.

adaptive ac control in final design
Dedicated control panel for passengers

Passengers didn’t have easy-to-access controls like drivers with steering wheel buttons. I came up with the passenger panel that brings music control at their fingertips. They can also save their preferred seat positions to their accounts.

passenger panel entry point and layout

Smart Suggestions to increase interaction efficiency

I created a journey map to understand what tasks a driver typically conducts, leading to two key findings:
1. Driver repeatedly use less than 3 apps in a driving session
2. Many tasks are predictable under certain conditions.

I proposed INFO FLOW to provide contextual suggestions by surfacing apps and services when drivers need them. INFO FLOW is a card feed consisting recommended app or service. This leads to more efficient interaction, and therefore safer driving because drivers can keep their eyes on the road.

Because different parts of the system are involved in this feature, our design team ideated together based on my concept. Then I established the triggering conditions of smart recommendations. Because of confidentiality, some descriptions are placeholders.

INFO FLOW anatomy
Cards follow a reverse chonological order, new cards appear from the top.
App cards
An app card will be added to the flow everytime an app is closed, so that users can access their recently used app from INFO FLOW with one step. When the system starts, Phone and Music app cards will be populated as the default state.
Music app card
phone app card
2. suggestion cards
Knowing context info, system can predict user needs and provide suggestion cards to allow drivers to finish a task with one single tap.
suggestion based on context info
INFO flow: app and service shortcuts

Given the confidentiality of some not-shipped designs, not all of my work was included. Below is a screenshot of a design document I created, which now becomes the template of the team. The document was used to communicate between interaction designers, visual designers, and engineers. Many features have multiple status, and the interaction can be affected by multiple factors including user intervention, car status, or auto dismiss. Therefore descriptive text and illustrations are important.

i created the documentation template and wrote the system UI interaction Design document
visual design

Collaborating with visual designer and developer to create hi-fidelity prototypes

I worked with a visual designer to deliver a modern and futuristic look.

final UI shipped with product, visual created by visual designer
testing and iterating
Testing in real-world condition revealed critical usability issues
Early prototyping and testing were done on display units instead of in a physical car environment. This is because we didn’t have a car prototype unit until our initial design concept was finished. As soon as we had a test car, we did a more holistic usability testing of the system, including static tests and active tests (driving on real roads). I used Keynote to document UI issues, with screenshots or recordings, which was used to debrief the engineer or designer responsible with fixing them.
I conducted design review in the lab vs. in the field
Usability issue due to hardware constraints
i used keynote to document usability issues
One critical issue we found was that the physical surface underneath the display hinders hands from interacting with the bottom part of the display, where Dock bar was located.
i recreated reachability map with ergonomics considerations
Iteration to accomodate limitation
Based on the reachability map, I reconstructed the placement of system UI. The main change was to move dock bar from bottom to left, which also affects some other elements.
Some other usability issues
We identified some issues that wouldn’t be exposed if we were not driving and testing in a real road environment.
some issues that I discovered in active tests

Intelligent, inclusive & safe

Experience caters to passengers

The system was designed with the consideration that the vehicle is not only a transportation tool for drivers, but also a space to be shared by passengers.
Smart Recommendation
Info Flow helps users interact with the system efficiently so that they can have more attention on the road.
Easily Reachable UI
Elements that need to be frequently accessed are carefully placed within areas that can be easily reached.
result & reflection

After launch, the car was positively reviewed by Chinese medias especially because of its smart features. It got the highest smart car rating from IVISTA. The new Xmart OS won the iF design awards 2022. P7 was the best-selling EV in Chinese market for 3 consecutive quarters in 2021.

On the personally level, the major takeaway is to learn to design in the context. Designing for automobile is not only considering about the screen, but also about the context the users are in. If they are driving, is it safe for them to use this feature? My cross-screen info flow design didn’t make the final cut because after discussion with PM, I realized that the complexity was so high that it could confuse users.

On the organization level, I realized software R&D should be synchronized with hardware team because system features could be infeasible because of hardware limitation. Knowing hardware limitation early on and testing regularly in its physical form are very critical.

No data collection

Last Updated - Jul 25, 2022