Download MODALIZER+What is DICOM Storage Commitment Service and why is it needed
Storage Commitment Data Flow
All together SCM is pretty straight forward. All we need to do is to send a list of the instances and get back a reply saying which are in the PACS database and which are not, and that’s exactly how it works. Well, almost.
The above diagram that I made hopefully explains it all. On the left there’s the SCM request. It is sent using N-ACTION command with a DICOM dataset that contains:
- A Transaction UID, identifying this commit request
- A referenced SOP Sequence, DICOM Tag (0008,1199) with a list of SOP Class UID’s and SOP Instance UID’s we request to commit.
- The transaction UID from the request.
- A list of succeeded instances
- If not all were ok, a list of failed instances
The Timing of the Storage Commitment Result
Getting the SCM result can sometimes be tricky. Maybe a similar design paradigm to the one described earlier led to this. Maybe, the SCP can’t answer immediately and it needs to think about it, queue the request for some batch process, check the database, compose a reply, queue it for sending and so on. Instead of just getting the results immediately in a response, DICOM lets the SCM SCP the freedom to decide when and how to answer. The SCP should send us back a N-EVENT REPORT command with the result and this result can arrive in one of three ways:- The SCP can Send the N-EVENT-REPORT on the same association that the SCU initiated or
- The SCP can start another association to the SCU and send the N-EVENT-REPORT, or
- The next time the SCU starts an association the SCP can send the N-EVENT-REPORT immediately after the association negotiation phase.
Implementing DICOM Storage Commit SCU with RZDCX
Because the result may be coming on another association, we better have an accepter running. We don’t have to but it’s a good idea because some SCP’s will not do it any other way. In DCXACC there’s a callback named OnCommitResult that hands out the Transaction UID’s and the succeeded and failed instances lists. We can run this accepter on a different process, on a different thread or on the same thread as you’ll see in the example. To send the request you can either call CommitFiles or CommitInstances. If you call the first, RZDCX will open each file, extract the SOP Class UID and SOP Instance UID and build the request dataset. If you use CommitInstances than you have to provide the list. CommitFiles is handier though because you’re not going to delete these files before you got the commit result anyhow. These two methods will not wait for the result and hang out immediately. There’s also CommitFilesAndWaitForResult and its pair CommitInstancesAndWaitForResult that wait for a while before hanging out and gives you the same out parameters as OnCommitResult does. If your PACS support that, these would be easier. Decent PACS should have a flag that controls this behavior for every AE Title and let you select between the ways that the results are sent back to the SCU. Here’s a single threaded example. What is done here is to set an accepter and start it, then send the request, then wait for the result on the accepter. I don’t recommend doing it this way but it kind of cool as an example. The best way I think is to have the accepter run independently on another thread or process that if you implement Storage SCP (which you probably do in order to get instances back on Q/R) also handles incoming C-STORE’s.C# Test Code
The example code this time is directly from my nUnit test suite. You can download RZDCX Examples with this and other examples. All these examples do basically the same: Create some DICOM Test Files, Save them to disk, Send them using C-STORE, Check that they were stored with Storage Commitment and wait for the result.
public void CommitFilesSameThread()
{
// Create
test files
String
fullpath = "SCMTEST";
Directory.CreateDirectory(fullpath);
CommonTestUtilities.CreateDummyImages(fullpath,
1, 1);
// Send
test files
string
MyAETitle = System.Environment.GetEnvironmentVariable("COMPUTERNAME");
DCXREQ
r = new DCXREQ();
string
succeededInstances;
string
failedInstances;
r.Send(MyAETitle, IS_AE, IS_Host,
IS_port, fullpath + "\\SER1\\IMG1",
out succeededInstances, out failedInstances);
Assert.That(failedInstances.Length
== 0);
Assert.That(succeededInstances.Length
> 0);
// Commit
files and wait for result on separate association for 30 seconds
SyncAccepter
a1 = new SyncAccepter();
r.CommitFiles(MyAETitle, IS_AE,
IS_Host, IS_port, fullpath + "\\SER1\\IMG1");
a1.WaitForIt(30);
if
(a1._gotIt)
{
//
Check the result
Assert.True(a1._status,
"Commit result is not success");
Assert.That(a1._failed_instances.Length
== 0);
DCXOBJ
obj = new DCXOBJ();
obj.openFile(fullpath + "\\SER1\\IMG1");
string
sop_class_uid = obj.getElementByTag((int)DICOM_TAGS_ENUM.sopClassUid).Value.ToString();
string
instance_uid = obj.getElementByTag((int)DICOM_TAGS_ENUM.sopInstanceUID).Value.ToString();
Assert.AreEqual(a1._succeeded_instances,
sop_class_uid + ";" + instance_uid
+ ";");
}
else
Assert.Fail("Didn't get commit result");
/// Cleanup
Directory.Delete(fullpath,
true);
}
Here’s the sync accepter:
class SyncAccepter
{
public
bool _gotIt = false;
public
bool _status = false;
public
string _transaction_uid;
public
string _succeeded_instances;
public
string _failed_instances;
public
DCXACC accepter;
public
string MyAETitle;
public
void accepter_OnCommitResult(
bool
status,
string
transaction_uid,
string
succeeded_instances,
string
failed_instances)
{
_gotIt = true;
_status = status;
_transaction_uid =
transaction_uid;
_succeeded_instances =
succeeded_instances;
_failed_instances =
failed_instances;
}
public
SyncAccepter()
{
accepter = new DCXACC();
accepter.OnCommitResult += new IDCXACCEvents_OnCommitResultEventHandler(accepter_OnCommitResult);
MyAETitle = System.Environment.GetEnvironmentVariable("COMPUTERNAME");
accepter.WaitForConnection(MyAETitle, 104, 0);
}
public
bool WaitForIt(int
timeout)
{
if
(accepter.WaitForConnection(MyAETitle, 104, timeout))
return
accepter.WaitForCommand(timeout);
else
return
false;
}
}
And here’s the example that waits for the results on the
same association.
public void CommitFilesAndWaitForResultOnSameAssoc()
{
bool
status = false;
bool
gotIt = false;
String
fullpath = "SCMTEST";
Directory.CreateDirectory(fullpath);
CommonTestUtilities.CreateDummyImages(fullpath,
1, 1);
string
succeededInstances;
string
failedInstances;
string
MyAETitle = System.Environment.GetEnvironmentVariable("COMPUTERNAME");
DCXREQ
r = new DCXREQ();
r.OnFileSent += new IDCXREQEvents_OnFileSentEventHandler(OnFileSent);
r.Send(MyAETitle, IS_AE, IS_Host,
IS_port, fullpath + "\\SER1\\IMG1",
out succeededInstances, out failedInstances);
DCXOBJ
obj = new DCXOBJ();
obj.openFile(fullpath + "\\SER1\\IMG1");
string
sop_class_uid = obj.getElementByTag((int)DICOM_TAGS_ENUM.sopClassUid).Value.ToString();
string
instance_uid = obj.getElementByTag((int)DICOM_TAGS_ENUM.sopInstanceUID).Value.ToString();
string
transactionUID = r.CommitFilesAndWaitForResult(MyAETitle, IS_AE, IS_Host,
IS_port, fullpath + "\\SER1\\IMG1",
5, out
gotIt, out status, out
succeededInstances, out failedInstances);
Directory.Delete(fullpath,
true);
Assert.True(status,
"Commit result is not success");
Assert.That(failedInstances.Length
== 0);
Assert.AreEqual(succeededInstances,
sop_class_uid + ";" + instance_uid
+ ";");
}
And here’s the common test utilities class in case you need
it
using System;
using
System.Collections.Generic;
using
System.Text;
using rzdcxLib;
using System.IO;
namespace rzdcxNUnit
{
class CommonTestUtilities
{
public static string
TestPatientName
{
get
{ return "John^Doe";
}
}
public static string
TestPatientID
{
get
{ return "123765";
}
}
public static string
TestStudyInstanceUID
{
get
{ return "123765.1";
}
}
public static string
TestSeriesInstanceUID
{
get
{ return "123765.1.1";
}
}
///
/// Create to series with 4 images each of test images
///
///
Root directory to put
the files in
///
public static unsafe List<String>
CreateDummyImages(String path)
{
return
CreateDummyImages(path, 4, 2, "", false);
}
public static unsafe List<String>
CreateDummyImages(String path, int numSeries, int
numImagesPerSeries)
{
return
CreateDummyImages(path, numSeries, numImagesPerSeries, "",
false);
}
public static unsafe List<String>
CreateDummyImages(String path, int numSeries, int
numImagesPerSeries, String suffix)
{
return
CreateDummyImages(path, numSeries, numImagesPerSeries, suffix, false);
}
///
/// Create a set of test images
///
///
Root directory to put
the files in
///
How many series
to create
///
How
many image files per series
///
filename suffix to
use
///
public static unsafe List<String>
CreateDummyImages(String path, int numSeries, int
numImagesPerSeries, String suffix, bool long_uid_names)
{
List<String> filesList = new
List<string>();
const
int ROWS = 64;
const
int COLUMNS = 64;
const
int SAMPLES_PER_PIXEL = 1;
const
string PHOTOMETRIC_INTERPRETATION = "MONOCHROME2";
const
int BITS_ALLOCATED = 16;
const
int BITS_STORED = 12;
const
int RESCALE_INTERCEPT = 0;
DCXOBJ
obj = new DCXOBJ();
/// Create an element pointer to place in the object for every
tag
DCXELM
el = new DCXELM();
/// Set Hebrew Character Set
el.Init((int)DICOM_TAGS_ENUM.SpecificCharacterSet);
el.Value = "ISO_IR
192";
/// insert the element to the object
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.Rows);
el.Value = ROWS;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.Columns);
el.Value = COLUMNS;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.SamplesPerPixel);
el.Value = SAMPLES_PER_PIXEL;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.PhotometricInterpretation);
el.Value =
PHOTOMETRIC_INTERPRETATION;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.BitsAllocated);
el.Value = BITS_ALLOCATED;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.BitsStored);
el.Value = BITS_STORED;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.HighBit);
el.Value = BITS_STORED - 1;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.PixelRepresentation);
el.Value = 0;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.WindowCenter);
el.Value = (int)(1 << (BITS_STORED - 1));
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.WindowWidth);
el.Value = (int)(1 << BITS_STORED);
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.RescaleIntercept);
el.Value = (short)RESCALE_INTERCEPT;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.RescaleSlope);
el.Value = 1;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.GraphicData);
el.Value = "456\\8934\\39843\\223\\332\\231\\100\\200\\300\\400";
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.PixelData);
el.Length = ROWS * COLUMNS *
SAMPLES_PER_PIXEL;
el.ValueRepresentation = VR_CODE.VR_CODE_OW;
ushort[]
pixels = new ushort[ROWS
* COLUMNS];
for
(int y = 0; y < ROWS; y++)
{
for
(int x = 0; x < COLUMNS; x++)
{
int
i = x + COLUMNS * y;
pixels[i] = (ushort)(((i) % (1 << BITS_STORED)) -
RESCALE_INTERCEPT);
}
}
fixed
(ushort* p = pixels)
{
UIntPtr
p1 = (UIntPtr)p;
el.Value = p1;
}
obj.insertElement(el);
// Set
identifying elements
el.Init((int)DICOM_TAGS_ENUM.PatientsName);
el.Value = CommonTestUtilities.TestPatientName;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.patientID);
el.Value = TestPatientID;
obj.insertElement(el);
String
study_uid = "123765.1";
el.Init((int)DICOM_TAGS_ENUM.studyInstanceUID);
el.Value = CommonTestUtilities.TestStudyInstanceUID;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.StudyID);
el.Value = 1;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.sopClassUid);
el.Value = "1.2.840.10008.5.1.4.1.1.7";
// Secondary Capture
obj.insertElement(el);
for
(int seriesid = 1; seriesid <= numSeries;
seriesid++)
{
String
series_uid = study_uid + "." +
seriesid;
el.Init((int)DICOM_TAGS_ENUM.seriesInstanceUID);
el.Value = series_uid;
obj.insertElement(el);
string
series_path = "";
if
(long_uid_names)
series_path = path + "\\" + series_uid;
else
series_path = path + "\\SER" + seriesid;
Directory.CreateDirectory(series_path);
el.Init((int)DICOM_TAGS_ENUM.SeriesNumber);
el.Value = seriesid;
obj.insertElement(el);
for
(int instanceid = 1; instanceid <=
numImagesPerSeries; instanceid++)
{
String
instance_uid = series_uid + "." +
instanceid;
el.Init((int)DICOM_TAGS_ENUM.sopInstanceUID);
el.Value = instance_uid;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.InstanceNumber);
el.Value = instanceid;
obj.insertElement(el);
/// Save it
///
String
filename = "";
if(long_uid_names)
filename = series_path
+ "\\" + instance_uid + suffix;
else
filename = series_path
+ "\\IMG" + instanceid + suffix;
obj.saveFile(filename);
filesList.Add(filename);
}
}
return
filesList;
}
// The unsafe
keyword allows pointers to be used within the following method:
public static unsafe void Copy(byte* pSrc,
int srcIndex, byte[]
dst, int dstIndex, int
count)
{
if
(pSrc == null || srcIndex < 0 ||
dst == null
|| dstIndex < 0 || count < 0)
{
throw
new System.ArgumentException();
}
//int
srcLen = src.Length;
//int
dstLen = dst.Length;
//if
(srcLen - srcIndex < count || dstLen - dstIndex < count)
//{
// throw new System.ArgumentException();
//}
// The
following fixed statement pins the location of the src and dst objects
// in
memory so that they will not be moved by garbage collection.
fixed
(byte*
pDst = dst)
{
byte*
ps = pSrc;
byte*
pd = pDst;
//
Loop over the count in blocks of 4 bytes, copying an integer (4 bytes) at a
time:
for
(int i = 0; i < count / 4; i++)
{
*((int*)pd)
= *((int*)ps);
pd += 4;
ps += 4;
}
//
Complete the copy by moving any bytes that weren't moved in blocks of 4:
for
(int i = 0; i < count % 4; i++)
{
*pd = *ps;
pd++;
ps++;
}
}
}
}
}
Summary
Let’s wrap it all up:- There’s DICOM service called Storage Commitment (SCM) that gets a list of instances and gives back which are stored with our peer and can be safely deleted from our local disk and which are not and we better send them over again.
- The SOP Class UID of Storage Commitment is 1.2.840.10008.1.20.1
- The request is sent via N-ACTION with the well known SOP Instance UID
- The result is received via N-EVENT-REPORT
- The result can come on a separate association.
- If we want to give the PACS a chance to send the result on the same association, we should at least wait for it to come for couple of seconds after sending the request.
Q&A
Q: Should the list of instances in the commit request be identical to the group of instances we sent in the association that stored the files?
A: No. In DICOM there's no contextual meaning to the association.Q: If some files failed to commit, should I send all files again or just the ones that failed?
A:I would recommend sending just the one that failed.Q: What if I didn't get the result?
A: Send a Storage Commit Request again.Q: How can I know the reason for the failure from the commit result?
A: You can't but from the C-STORE command response you can. There's a status there and sometimes additional explanation attributes. Read the log.
Why not to implement it the way described above
One last thing I owe you. You can say that the command succeeded but the file is dead, ha? After all, we are in a hospital right, the operation was successful but the patient died? anyone? Never mind. Implementations that just store the file on disk, say OK and later parse it and find out that it’s wrong and can’t actually process it simply don’t have any way to tell the client what was the problem with the file. They lose the only chance to respond properly in the C-STORE Response. If you fear that processing the registration of a new instance during the C-STORE will delay the communication so you better review your registration process. Maybe your DICOM parser is too slow or your database connection is not optimized.<- Prev - Modality Performed Procedure Step Pixel Data - Next ->
Excellent article.
ReplyDeleteThank you.
Ariel T.
Hello,
ReplyDeleteThanks for this great tutorial. I would like to hear your opinion about the following case:
I am implementing a DICOM router, it receives a full study very fast in a local device, and then uploads it to a repository on the cloud.
So, I am wondering, when is the best moment to confirm the arrival of the image/s? at router or cloud repository? What to answer when the images are still in the router buffer?
Thanks in advance,
Jaime O.
Hi Jamie
DeleteThats an interesting question.
I would suggest to check what IHE has to say about it. The key to search would be 'shared image archive' in the XDS-I integration profile.
I'll do this check as well and re-reply.
The way I would do it is to reply success once it's stored on the proxy regardless of the upload progress.
Otherwise, if you reply failed, the modality will send the study again and this is not what you want.
Regards
Roni
Excellent Article!
ReplyDeleteThanks! Now I have a idea on how the whole SC works!
Cheers,
Navin.
Excellent Articles.
ReplyDeleteHey,
I am using DCM4CHEE as my PACS. It supports the Storage Commit but I am not able to figured out how the SCP can start another association to the SCU and send the N-EVENT-REPORT using DCM4CHEE. do you have any idea. by default, it support two way communication, means send the N-Action Report using new association. Do you have any idea hwo to enable one way communication?
Thanks,
Deepak
Thanks. Unfortunately I'm not a Java guy. Most of our work is in C# and C++. I've used DCM4CHEE maybe once or twice so can't really help. Sorry.
DeleteGreat explanation - but i could not find anything in the dicom specification (2011) about the timing that is described as follows:
ReplyDelete"The next time the SCU starts an association the SCP can send the N-EVENT-REPORT immediately after the association negotiation phase."
Can you give us the part and chapter from the dicom specification where this kind of protocol is specified ?
TIA
Manfred from Austria
This is from IHE Cardiology
DeleteSo, the third option is actually not as described. According to what is said in the IHE cardiology transaction (Storage Commitment [CARD-3]) the modality (SCU) opening an Association (message channel) for Storage Commitment shall serve as a trigger for the Image Manager (SCP) to open a separate Association to the Modality to send the queued N-Event Report messages. Therefore, it is a separate association, not the one that is just being opened. Any thoughts?
DeleteTrue.
DeleteI have a doubt
ReplyDeletein which DIMSE Request the 1.2.840.10008.1.20.1 value is filled and send
N-ACTION or in N-EVENT-REPORT?
Hi Surjith,
ReplyDeleteSOP Class UID : 1.2.840.10008.1.20.1 (Storage Commitment Push Model SOP Class) is part of both 'N-ACTION' and 'N-EVENT-REPORT' Requests.
In N- ACTION it is mentioned as "Requested SOP Class UID" (0000, 0003)
In N- EVENT-REPORT' it is mentioned as "Affected SOP Class UID" (0000,0002).
Roni Please correct me if I am wrong :).
Hey!
ReplyDeletecan transmission of files happen directly between 2 devices which are dicom-interfaced? or do we need a server for this? if a server is needed, then do we need to write a separate application code for it or can the existing server deal with the dicom format files?
DICOM communication Always takes place between two applications. In most cases, applications that are capable to retrieve DICOM files from a PACS using the DICOM C-MOVE command will also be able to receive unsolicited C-STORE commands from other applications.
DeleteHi Roni,
ReplyDeleteYour blog really helped me! Thanks for sharing the information. I could not download the full code. Can you send me?
Rotem Cohen
The link is ok: http://downloads.roniza.com/downloads/rzdcx/Examples/RZDCX_2003_EXAMPLE-APPLICATIONS.zip
DeleteHi,
ReplyDeleteThank you for your explanation.
I have a question, how does the SCP know that sent files are well stored? Is it with a script or it can acces to database?
Hi,
DeleteWell, the SCP is the one that received the files and is responsible for storing them. If it can’t tell then who can?
Hi,
ReplyDeleteone thing is unclear to me about: "The result can come on a separate association"
As I understand it, my app can also wait for association requests from other computer. My app is kind of server then.
Now my app as client makes request to other computer (for which I have pre-configured ip:port), and asks for Storage Commitment status. But if the results don't come in actual association (my app closes it), it's promised that "result can come on a separate association". Possibly initiated by other party. Correct?
But how does the other computer know my IP:port to which it will attempt to connect to, make association and deliver results? How does the other computer even track my request and know it's my app again to which it will deliver the results through n-event-report?
Thanks.
Hi Mike,
DeleteThe other app (the storage commitment SCP has to be ‘pre configured’ as well with your AR title, ip address and port. Then it can resolve the AE title from your request to the IP address and port to initiate the communication with and send the result.
Regards,
Roni
Hi,
ReplyDeleteSorry if this isnt the right place to ask about this situation i have in my hands.
I will need to work with multiple pac servers, one of them is a "central", that receive data from all the other servers, basically:
Central Pac Server
/ | \
PacServer1 PacServer2 PacServer3
| | |
workstation workstation workstation
What i would like to know is if such thing is possible. And if such a thing is recommended.
There are all kind of architectures. It's possible.
Delete