Removing obsolete pages from search engine indexation using the robots.txt was already discussed in
The related part of the discussion began with this post.
@darknao I expect you currently have to focus the GitLab migration, but am I right to assume that it is already planned to remove unsupported/obsolete docs pages from search engine indexation anyway?