In multi‐state models, the expected length of stay (ELOS) in a state is not a straightforward object to relate to covariates, and the traditional approach has instead been to construct regression models for the transition intensities and calculate ELOS from these. The disadvantage of this approach is that the effect of covariates on the intensities is not easily translated into the effect on ELOS, and it typically relies on the Markov assumption. We propose to use pseudo‐observations to construct regression models for ELOS, thereby allowing a direct interpretation of covariate effects while at the same time avoiding the Markov assumption. For this approach, all we need is a non‐parametric consistent estimator for ELOS. For every subject (and for every state of interest), a pseudo‐observation is constructed, and they are then used as outcome variables in the regression model. We furthermore show how to construct longitudinal (pseudo‐) data when combining the concept of pseudo‐observations with landmarking. In doing so, covariates are allowed to be time‐varying, and we can investigate potential time‐varying effects of the covariates. The models can be fitted using generalized estimating equations, and dependence between observations on the same subject is handled by applying the sandwich estimator. The method is illustrated using data from the US Health and Retirement Study where the impact of socio‐economic factors on ELOS in health and disability is explored. Finally, we investigate the performance of our approach under different degrees of left‐truncation, non‐Markovianity, and right‐censoring by means of simulation