According to Wikipedia, the 2nd Earl of Rosslyn was 'Acting Grand Master of the Grand Lodge of Scotland, on behalf of King George IV. '
I assume that's masonic regalia he's wearing in his portrait.
https://en.wikipedia.org/wiki/James_...arl_of_Rosslyn