In a food processor (or by hand), finely chop the parsley, garlic, and oregano. Set aside.
In a skillet, over medium heat, cook the bacon until crispy, 4 to 5 minutes. Transfer to a paper towel-lined plate. Remove all but 1 tablespoon of the drippings.
Add 1 tablespoon of the butter, the celery hearts, ¼ teaspoon salt, and ⅛ teaspoon pepper. Cook for 2 minutes, tossing to coat well.
Add the chicken broth, cover, and simmer until the celery is tender, about 20 minutes. Remove from heat, add the reserved parsley mixture, and toss well.
Meanwhile, bring 4 cups of water to a boil. Slowly whisk in the polenta. Add ¾ teaspoon salt and cook, stirring frequently, for 5 minutes. Stir in ½ cup of the Parmesan and remaining tablespoon butter and mix well.
Divide the polenta among bowls and top with the celery and bacon. Spoon the cooking liquid on top. Sprinkle with the remaining tablespoon Parmesan.