I laughed harder than I should have.
The best part is that according to YNB’s Twitter is that the only cast member who knew about Chevy Chase’s cameo was Joel McHale. The rest of the cast didn’t know until they saw the finished episode as a group.
Dan Harmon said the cast were mislead in the table-read thanks to a scene between Jeff and Starburns that was written, but never intended for filming, that was inserted in the scripts.
Apparently, whatever happened during S4 was bad enough that as part of Chevy’s separation agreement, he was legally barred from every setting foot on the set. So he literally filmed this scene all by himself against a green screen.
Read more about how the Chevy Chase thing happened at the source.