It follows the IS takeover of the town of Zumar and two nearby oilfields from Kurdish Peshmerga forces on Saturday.
IS seized large parts northern Iraq from government control in a major offensive in June.
The UN special envoy to Iraq, Nickolay Mladenov, said that a "humanitarian tragedy is unfolding in Sinjar".
"The United Nations has grave concerns for the physical safety of these civilians" he said.
"The humanitarian situation of these civilians is reported as dire, and they are in urgent need of basic items including food, water and medicine" he added.
The UN said many of those who fled are in exposed areas in mountains near the town.
Many of those in Sinjar are believed to have fled from earlier IS advances in northern Iraq.
The town is home to a large community of Kurdish Yazidis, whom IS consider heretical.
Two Yazidi shrines have reportedly been destroyed in the town.
Kurdish military forces, known as the Peshmerga, were also forced to retreat from the nearby town of Zumar on Saturday after a militant offensive.
Kurdish forces had held the town since the Iraqi army retreated from the are in June.
Eyewitnesses said militants also seized control of two small oilfields near Zumar.
IS already controls several other oil installations in northern Iraq, which are believed to fund its activities.