About Ashkrit Sharma
Power of Java MemoryMapped File
In JDK 1.4 an interesting feature of Memory mapped file was added to Java, which allows to map any file to OS memory for efficient reading. A memory mapped file can be used to develop an IPC type of solution. This article is an experiment with memory mapped file to create IPC.
Some details about Memory Mapped File, definition from WIKI
A memory-mapped file is a segment of virtual memory which has been assigned a direct byte-for-byte correlation with some portion of a file or file-like resource. This resource is typically a file that is physically present on-disk, but can also be a device, shared memory object, or other resource that the operating system can reference through a file descriptor. Once present, this correlation between the file and the memory space permits applications to treat the mapped portion as if it were primary memory.
Sample Program
Below we have two Java programs, one is a writer and the other is a reader. The writer is the producer and tries to write to Memory Mapped file, the reader is the consumer and it reads messages from the memory mapped file. This is just a sample program to show you the idea, it does’t handle many edge cases but it is good enough to build something on top of a memory mapped file.
MemoryMapWriter
import java.io.File;import java.io.FileNotFoundException;import java.io.IOException;import java.io.RandomAccessFile;import java.nio.MappedByteBuffer;import java.nio.channels.FileChannel;public class MemoryMapWriter { public static void main(String[] args) throws FileNotFoundException, IOException, InterruptedException { File f = new File("c:/tmp/mapped.txt"); f.delete(); FileChannel fc = new RandomAccessFile(f, "rw").getChannel(); long bufferSize=8*1000; MappedByteBuffer mem =fc.map(FileChannel.MapMode.READ_WRITE, 0, bufferSize); int start = 0; long counter=1; long HUNDREDK=100000; long startT = System.currentTimeMillis(); long noOfMessage = HUNDREDK * 10 * 10; for(;;) { if(!mem.hasRemaining()) { start+=mem.position(); mem =fc.map(FileChannel.MapMode.READ_WRITE, start, bufferSize); } mem.putLong(counter); counter++; if(counter > noOfMessage ) break; } long endT = System.currentTimeMillis(); long tot = endT - startT; System.out.println(String.format("No Of Message %s , Time(ms) %s ",noOfMessage, tot)) ; }} MemoryMapReader
import java.io.File;import java.io.FileNotFoundException;import java.io.IOException;import java.io.RandomAccessFile;import java.nio.MappedByteBuffer;import java.nio.channels.FileChannel;public class MemoryMapReader { /** * @param args * @throws IOException * @throws FileNotFoundException * @throws InterruptedException */ public static void main(String[] args) throws FileNotFoundException, IOException, InterruptedException { FileChannel fc = new RandomAccessFile(new File("c:/tmp/mapped.txt"), "rw").getChannel(); long bufferSize=8*1000; MappedByteBuffer mem = fc.map(FileChannel.MapMode.READ_ONLY, 0, bufferSize); long oldSize=fc.size(); long currentPos = 0; long xx=currentPos; long startTime = System.currentTimeMillis(); long lastValue=-1; for(;;) { while(mem.hasRemaining()) { lastValue=mem.getLong(); currentPos +=8; } if(currentPos < oldSize) { xx = xx + mem.position(); mem = fc.map(FileChannel.MapMode.READ_ONLY,xx, bufferSize); continue; } else { long end = System.currentTimeMillis(); long tot = end-startTime; System.out.println(String.format("Last Value Read %s , Time(ms) %s ",lastValue, tot)); System.out.println("Waiting for message"); while(true) { long newSize=fc.size(); if(newSize>oldSize) { oldSize = newSize; xx = xx + mem.position(); mem = fc.map(FileChannel.MapMode.READ_ONLY,xx , oldSize-xx); System.out.println("Got some data"); break; } } } } }} Observation
Using a memory mapped file can be a very good option for developing Inter Process communication, throughput is also reasonably well for both produce & consumer. Performance stats by run producer and consumer together:
Each message is one long number
Produce – 10 Million message – 16(s)
Consumer – 10 Million message 0.6(s)
A very simple message is used to show you the idea, but it can be any type of complex message, but when there is complex data structure then serialization can add to overhead. There are many techniques to get over that overhead. More in next blog.
Source : http://www.javacodegeeks.com/2013/05/power-of-java-memorymapped-file.html
