Nasdaq OMX Group Inc. will form a joint venture with SharesPost Inc. to establish a marketplace for private companies, Reuters reports.