non cached memory for user space DMA

I am trying to write a MAC driver for Arm 11 MPcore in 2.6.23. The DMA'ble memory required is supposed to be non-cached for performance reasons.

A chunk of memory(~ 6MB) is allocated at startup using bigphys area patch. This physical area in the RAM is mmap'd to userspace as follows and thereafter carved into network buffers:

fd = open("/dev/mem", O_RDWR|O_SYNC);
uptr = mmap(0, sz, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_LOCKED,fd, pptr);

Inspite of using the O_SYNC flag the user space mapping is cached and the DMA'd data is not visible. I tried writing up my own mmap and marked the vma non-cached without any results:

size_t size = vma->vm_end - vma->vm_start;
unsigned long offset = vma->vm_pgoff << PAGE_SHIFT;

vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
if (remap_pfn_range(vma, vma->vm_start, offset, size, vma->vm_page_prot)) {
return -EAGAIN;

I found that the bigphys patch allocated memory from bootmem and bootmem maps the memory as MT_MEMORY. Is there a way I could reduce bootmem allocation at startup (using kernel boot cmd mem=x ?) ?

The idea is to map the rest of the available memory as MT_DEVICE(assuming it removes the BUFFERABLE & CACHEABLE bits while mapping) so that the kernel mapping is also non-cacheable. Hopefully, this will ease the userspace mapping to non-cached as well ?

Deb Rupam Banerjee

Geek Guide
The DevOps Toolbox

Tools and Technologies for Scale and Reliability
by Linux Journal Editor Bill Childers

Get your free copy today

Sponsored by IBM

Upcoming Webinar
8 Signs You're Beyond Cron

Scheduling Crontabs With an Enterprise Scheduler
11am CDT, April 29th
Moderated by Linux Journal Contributor Mike Diehl

Sign up now

Sponsored by Skybot