The root of the difference between Sunnis and Shi'is is that a Shi'i believes that the Prophet appointed Imam 'Ali as his direct successor in all ways (political, spiritual, practical, etc), and a Sunni does not. (Even if the Sunni might consider Imam 'Ali more meritorious than the first three caliphs, or might consider Imam 'Ali as the Prophet's spiritual successor, etc.) And 'Eid al-Ghadir is essentially a celebration of the Prophet's final public declaration of Imam 'Ali as his successor.
Many Sunnis have a different understanding from Shi'is of what happened at Ghadir, both historically, in terms of the interpretation of the hadith of Ghadir, and in terms of the understanding of Qur'an 5:3.
Also, 'Eid al-Ghadir is specified as an 'Eid by Shi'i hadith. That is, Shi'is do not independently decide "let's celebrate 'Eid al-Ghadir" or "let's call the Day of Ghadir an 'Eid", but rather it is specified in hadith as an important 'Eid, and in fact the most important 'Eid.
However, Sunni hadith do not specify the day of Ghadir as an 'Eid. There is a narration in in Tarikh Baghdad which speaks of the "Day of Ghadir" and recommends fasting on that day. However, Tarikh Baghdad is not a major source of Sunni theology, and, in any case, it doesn't specifically say "'Eid al-Ghadir" (vol. 8, p. 284).
So, one could also say that both Shi'is and Sunnis are following what is in their hadith sources.
Of course there are exceptions especially in areas where there are blurrier lines between Sunni and Shi'i practice or a mixed heritage of Sunnis and Sh'ism.
