Abstract: In order to improve the feature robustness of the traditional object tracking algorithm, an improved algorithm based on background-weighted spatial histogram was proposed. It combines the background-weighted histogram and spatial histogram and exploits the advantages of both features, which distinctively improves the feature robustness. The proposed algorithm was implemented under the framework of Mean Shift tracking and obtained a stable tracking. Experimental results indicate that the proposed algorithm can well track the object and obtains at least 80% decreasing of the mean center location error than that of the traditional algorithms.