Like his own twins, Harry is also a twin with actor brother Luke.
The top couple were first spotted breaking up with their babies in tow in Devon last week.
This is the first time they have been seen since the babies were recently born.
They moved from London to Devon when they found out Holliday was pregnant last year.
Borgia Holliday actress, 33, and Harry have been in love since they first fell in love in 2015.
Today, the couple became parents for the first time.
Seen in pictures on MailOnline, the lovebirds enjoyed a discreet walk with their newborns.
“It was a beautiful and happy family outing,” a viewer told the newspaper.
“Even though both actors are on our TV screens all the time, no one has paid them any attention.
“But the twins received a lot of admiring comments from groups of old ladies. Mom and dad seemed delighted with this. “
They keep their romance low-key, despite being both famous in their own right.
Get the inside scoop on all your favorite celebrities with our daily newsletter delivered straight to your inbox for free.
You can register here.
Most recently, Holliday played Detective Inspector Rachel Carey in The Capture.
She has built a huge fan base with her role as a detective.
A second series for The Capture is believed to have been postponed due to her pregnancy, according to the newspaper.
She has had many notable roles including that of Lucrezia Borgia in The Borgias alongside Jeremy Irons.
In addition to roles on television, she had great success with her film titles.
Among her film titles, she played the cold leading lady Estrella in the 2011 adaptation of Great Expectations.