It was a bright start to the match with Linfield's Kieran Offord shooting just off target before, at the other end, Jack Malone had a good chance.
The away side believed they should have been awarded an early penalty when Chris Johns and Jordan Jenkins collided in the box when trying to connect with a poor back pass from Leahy, but their appeals were quickly waved away.
Linfield broke the deadlock on 18 minutes when Leahy held off both Ross Clarke and James Singleton to nod home Adam Frizzell's corner from the back post.
As the half progressed, chances grew few and far between with Johns called into action only in the final seconds to gather Malone's effort from range.
Opportunities remained at a premium after the restart with Billy Crellin forced to make a good save to thwart Millar's shot as the Blues pushed for a more comfortable lead.
The home side would extend their advantage after Offord was denied by Crellin before Millar latched onto the rebound and his effort found the back of the net after it struck the underside of the crossbar.
It was a tense final 10 minutes of the match with Glentoran able to set up an exciting end when Aaron McEneff's free-kick was headed into the path of Stewart to fire home
However, Stewart's celebrations were cut short when he was given his marching orders for elbowing Leahy in the final seconds of a cagey but entertaining 'Big Two' derby.
