Latest construction figures show a small decline of 0.2% in October, compared with the previous month.
The sector has said Brexit and the skills crisis are making things worse.
Commenting, Sarah McMonagle, Director of External Affairs at the FMB, said: “Rising costs for large and small construction firms are contributing to the slight drop in construction output.
“Recent government statistics show that in the past year, there were nearly 3,000 insolvencies in the construction industry.
“While wages are continuing to rise because of the ever-worsening skills crisis in our sector, firms are also feeling the pinch because of the rising cost of materials.
“The depreciation of sterling following the EU referendum has meant key materials have become more expensive.
“We are expecting material prices to continue to squeeze the margins of construction firms, with recent research from the Federation of Master Builders showing that almost 90% of builders believe that prices will continue to rise in the next six months.”
McMonagle concluded: “The future is still looking uncertain for the UK construction sector because the government has set out plans for a post-Brexit immigration system that would severely worsen the skills shortage.
“The government has said it will limit the number of low skilled workers entering the UK from the EU and further afield.
“This would include thousands of tradespeople, including bricklayers and carpenters, and these roles are ones that the construction sector relies on.
“More than 9% of the UK’s construction workers are from the EU and this rises to one third in London.
“It is therefore imperative that the government listens to the needs of the sector and delivers a post-Brexit immigration system that allows us to draw on essential migrant workers.
“Without this, we won’t be able to keep building at the current rate and construction output will continue to fall,” said McMonagle.