Heat the oil in a large skillet over medium-high heat. Cook the sausage until browned, 3 to 4 minutes. Add the shrimp, bell pepper, ½ teaspoon salt, and ¼ teaspoon pepper and cook until the shrimp are opaque, 3 to 4 minutes more. Add 2 tablespoons of the butter and cook until melted.
Meanwhile, in a medium saucepan, bring 3 cups of water to a boil. Whisk in the polenta and cook, whisking, until thickened, 3 to 4 minutes. Whisk in the remaining 2 tablespoons of butter, ½ teaspoon salt, and ⅛ teaspoon pepper. Serve with the shrimp mixture and sprinkle with the oregano.