I am trying to sum up values out of an datatable created out of an database query.
The times in MySQL DB are formatted like HH:mm:ss (ss is always 00)
I already had this running i just was unhappy with the result as for example 25 h as result resulted in 1.01:00:00 or 1 day and 1 hour when i wanted to receive 25 h as it is saved in the database, changing data type in mysql is not an option for me, as i have a bunch of calc's that need those as data type time.
Right now the code is not longer able to show the result in a textBox as I have more a less an error:
"Invalid use of aggregate function Sum () and Type: String."
As far as I understood I can not sum strings, just time and numbers, however I could do it before formatting time right when it gets pulled out of the db.
I did search in google for a good solution for a couple of days now, but the only options I see are thinks that I already tried around on my own (of cause tested again just more errors)
However here is the required code snippet:
MySqlCommand cmddttoday = conn.CreateCommand();
cmddttoday.CommandType = CommandType.Text;
cmddttoday.CommandText = "SELECT day, month, date_format (start, '%H:%i') start, date_format (stop, '%H:%i') stop, date_format (total, '%H:%i') total FROM arbeitszeiten WHERE username = '" + textBoxusername.Text + "' and day = '" + today + "' and month = '" + month + "' and year = '" + thisyear + "'";
cmddttoday.ExecuteScalar();
DataTable dttimetoday = new DataTable();
MySqlDataAdapter datadatoday = new MySqlDataAdapter(cmddttoday);
datadatoday.Fill(dttimetoday);
textBoxtimetoday.Text = dttimetoday.Compute("SUM(total)", "").ToString();
dttimetoday.Columns["day"].ColumnName = "Tag";
dttimetoday.Columns["month"].ColumnName = "Monat";
dttimetoday.Columns["start"].ColumnName = "Arbeit aufgenommen";
dttimetoday.Columns["stop"].ColumnName = "Arbeit abgelegt";
dttimetoday.Columns["total"].ColumnName = "Zeit gesammt";
Keep in mind, yes it does not work like this but without the format on total in the query it shows like explained before as 1.01:00:00
I would be very happy if we found a solution for this meanwhile annoying problem.
Thanks for your burning braincells ;)
I think using TIMESTAMPDIFF would be way easier to calculate the timedifference in minutes directly. From there you can use SUM to calculate the total of minutes. To get the hours you can then simply divide by 60. This would look like this:
SELECT SUM(TIMESTAMPDIFF(Minute,start, end)/60) from arbeitszeiten
Check out this Fiddle: Link
EDIT:
If thats the case, you have to use linq. The error you get is because the datatable column "total" is of the datatype string. Since you cant Sum a string this wont work. You cant convert withing the SUM function either since the Compute method requests a number inside of the brackets.
With linq you can do it like this:
textBoxtimetoday.Text = dttimetoday.AsEnumerable().Sum(r => Convert.ToDecimal(r.Field<string>(4))).ToString();
but you have to replace date_format (total, '%H:%i') total with date_format (total, '%H.%i') total
Related
I'm trying to read data from SAP ECC using Microsoft .NET. For this, I am using the SAP Connector for Microsoft .NET 3.0 Following is the code to retrieve the data, I'm getting the results too. However, I found that the exchange rate value is having a * if it exceeds 7 characters.
ECCDestinationConfig cfg = new ECCDestinationConfig();
RfcDestinationManager.RegisterDestinationConfiguration(cfg);
RfcDestination dest = RfcDestinationManager.GetDestination("mySAPdestination");
RfcRepository repo = dest.Repository;
IRfcFunction testfn = repo.CreateFunction("RFC_READ_TABLE");
testfn.SetValue("QUERY_TABLE", "TCURR");
// fields will be separated by semicolon
testfn.SetValue("DELIMITER", ";");
// Parameter table FIELDS contains the columns you want to receive
// here we query 3 fields, FCURR, TCURR and UKURS
IRfcTable fieldsTable = testfn.GetTable("FIELDS");
fieldsTable.Append();
fieldsTable.SetValue("FIELDNAME", "FCURR");
fieldsTable.Append();
fieldsTable.SetValue("FIELDNAME", "TCURR");
fieldsTable.Append();
fieldsTable.SetValue("FIELDNAME", "UKURS");
fieldsTable.Append();
fieldsTable.SetValue("FIELDNAME", "GDATU");
// the table OPTIONS contains the WHERE condition(s) of your query
// several conditions have to be concatenated in ABAP syntax, for instance with AND or OR
IRfcTable optsTable = testfn.GetTable("OPTIONS");
var dateVal = 99999999 - 20190701;
optsTable.Append();
optsTable.SetValue("TEXT", "gdatu = '" + dateVal + "' and KURST = 'EURX'");
testfn.Invoke(dest);
Values are as follows:
How to get the full value without any truncation?
You just ran into the worst limitation of RFC_READ_TABLE.
Its error is to return field values based on internal length and truncating the rest, rather than using the output length. TCURR-UKURS is a BCD decimal packed field of length 9,5 (9 bytes = 17 digits, including 5 digits after the decimal point) and an output length of 12. Unfortunately, RFC_READ_TABLE outputs the result on 9 characters, so a value of 105.48000- takes 10 characters is too long, so ABAP default logic is to set the * overflow character on the leftmost character (*5.48000-).
Either you create another RFC-enabled function module at SAP/ABAP side, or you access directly the SAP database (classic RDBMS connected to SAP server).
Just an addition to Sandra perfect explanation about this issue. Yes, the only solution here would be writing a custom module for fetching remote records.
If you don't want to rewrite it from scratch the simplest solution would be to copy RFC_READ_TABLE into Z module and change line 137
FIELDS_INT-LENGTH_DST = TABLE_STRUCTURE-LENG.
to
FIELDS_INT-LENGTH_DST = TABLE_STRUCTURE-OUTPUTLEN.
This solves the problem.
UPDATE: try BAPI_EXCHANGERATE_GETDETAIL BAPI, it is RFC-enabled and reads rates correctly. The interface is quite self-explanatory, the only difference is that date should be in native format, not in inverted:
CALL FUNCTION 'BAPI_EXCHANGERATE_GETDETAIL'
EXPORTING
rate_type = 'EURO'
from_curr = 'USD'
to_currncy = 'EUR'
date = '20190101'
IMPORTING
exch_rate = rates
return = return.
Use BBP_RFC_READ_TABLE. It is still not the best but it does one thing right which RFC_READ_TABLE did not: one additional byte for the decimal sign.
No need to go through all the ordeal if you only look for patching the decimal issue.
This is the sample code used with SAP connector for .NET, let it be helpful for someone who looks for the same. Thanks for all those who helped.
var RateForDate = 20190701;
ECCDestinationConfig cfg = new ECCDestinationConfig();
RfcDestinationManager.RegisterDestinationConfiguration(cfg);
RfcDestination dest = RfcDestinationManager.GetDestination("mySAPdestination");
RfcRepository repo = dest.Repository;
IRfcFunction sapFunction = repo.CreateFunction("RFC_READ_TABLE");
sapFunction.SetValue("QUERY_TABLE", "TCURR");
// fields will be separated by semicolon
sapFunction.SetValue("DELIMITER", ";");
// Parameter table FIELDS contains the columns you want to receive
// here we query 3 fields, FCURR, TCURR and UKURS
IRfcTable fieldsTable = sapFunction.GetTable("FIELDS");
fieldsTable.Append();
fieldsTable.SetValue("FIELDNAME", "FCURR");
//fieldsTable.Append();
//fieldsTable.SetValue("FIELDNAME", "TCURR");
//fieldsTable.Append();
//fieldsTable.SetValue("FIELDNAME", "UKURS");
// the table OPTIONS contains the WHERE condition(s) of your query
// here a single condition, KUNNR is to be 0012345600
// several conditions have to be concatenated in ABAP syntax, for instance with AND or OR
IRfcTable optsTable = sapFunction.GetTable("OPTIONS");
var dateVal = 99999999 - RateForDate;
optsTable.Append();
optsTable.SetValue("TEXT", "gdatu = '" + dateVal + "' and KURST = 'EURX'");
sapFunction.Invoke(dest);
var companyCodeList = sapFunction.GetTable("DATA");
DataTable Currencies = companyCodeList.ToDataTable("DATA");
//Add additional column for rates
Currencies.Columns.Add("Rate", typeof(double));
//------------------
sapFunction = repo.CreateFunction("BAPI_EXCHANGERATE_GETDETAIL");
//rate type of your system
sapFunction.SetValue("rate_type", "EURX");
sapFunction.SetValue("date", RateForDate.ToString());
//Main currency of your system
sapFunction.SetValue("to_currncy", "EUR");
foreach (DataRow item in Currencies.Rows)
{
sapFunction.SetValue("from_curr", item[0].ToString());
sapFunction.Invoke(dest);
IRfcStructure impStruct = sapFunction.GetStructure("EXCH_RATE");
item["Rate"] = impStruct.GetDouble("EXCH_RATE_V");
}
dtCompanies.DataContext = Currencies;
RfcDestinationManager.UnregisterDestinationConfiguration(cfg);
The question is why am I having a slow down at 25 million records. Is this a SQL Server config issue, code or both?
After approximately 25 million records loaded, in the routine pGetOHLCBetweenTwoDates(...) (see code below) the following line
SqlDataReader rdr = sqlCmd.ExecuteReader();
takes 20 times as long to load records from via T-SQL (SQL Server). It does not matter how many records are loaded per pass (50k, 100k,250k) it is always at around 25m records that the slow down occurs (see logs excerpts below).
It is not a resource issue, the system this is running on has 128 gb memory, dual 8 cores and SQL Server has 8 gig of memory avail, SQL Server is running locally, SQL Server was increased to 32 gig to see if that would fix the issue, it did not. I see the same thing on the SQL server profiler side 500-600 ms jumping to 10,000 - 30,000 ms.
idxLoadPos 25249999 - Elapsed time 00:00:00.9609748
idxLoadPos 25099999 - Elapsed time 00:00:00.5936540
idxLoadPos 24949999 - Elapsed time 00:00:00.5890105
idxLoadPos 24799999 - Elapsed time 00:00:11.5260435 <---<< Approx 25 million records loaded
idxLoadPos 24649999 - Elapsed time 00:00:10.9329704
idxLoadPos 24499999 - Elapsed time 00:00:11.2460554
Code:
public static int GetOHLCBetweenTwoDates(ref OHLCArray ohlcArray, int currentIndex, DateTime fromDate, DateTime toDate, DataFeedConfig dataFeedConfig)
{
// loads an array bottom up to match the standards in TA-LIBRARY in C Sharp
DateTime fDate = fromDate;
DateTime tDate = toDate;
int recordCount = SQL.GetCountBetweenTwoDates(fromDate, toDate, ohlcArray.TimePeriod, dataFeedConfig);
int maxReturnCount = LoadRecordCount;
//int maxReturnCount = 100000;
int getRecordCount = maxReturnCount;
int remainingRecordCount = recordCount;
int idxArrayLoadPosition = currentIndex;
Console.WriteLine("SQL - int - GetOHLCBetweenTwoDates from: {0}, to: {1}, getRecordCount: {2} ", fDate.ToString(), tDate.ToString(), getRecordCount.ToString());
while (fDate <= tDate & remainingRecordCount > 1) //0
{
getRecordCount = (remainingRecordCount < maxReturnCount) ? remainingRecordCount : maxReturnCount;
fDate = SQL.pGetOHLCBetweenTwoDates(ref ohlcArray, ref idxArrayLoadPosition, getRecordCount, fDate, tDate, ohlcArray.TimePeriod, dataFeedConfig);
remainingRecordCount -= (getRecordCount - 1); // no -1
Console.WriteLine("SQL - GetOHLCBetweenTwoDates from: {0}, to: {1}, getRecordCount: {2} ", fDate.ToString(), tDate.ToString(), getRecordCount.ToString());
}
idxArrayLoadPosition++;
if (idxArrayLoadPosition == -1) { idxArrayLoadPosition = 0; }
return idxArrayLoadPosition;
}
private static DateTime pGetOHLCBetweenTwoDates(ref OHLCArray ohlcArray, ref int idxArrayLoadPosition, int recordCount, DateTime fromDate, DateTime toDate, Enums.TimePeriodTypes tpt, DataFeedConfig dataFeedConfig)
{
DateTime returnDate = new DateTime(1901, 01, 01);
string database = "" + dataFeedConfig.Exchange + "." + dataFeedConfig.Issue + ".Data"; //string database = "[" + Exchange + "." + Issue + ".Data]";
string userName = dataFeedConfig.sqlLoginCredentials.SQLUserName;
string password = dataFeedConfig.sqlLoginCredentials.SQLPassword;
string server = dataFeedConfig.sqlLoginCredentials.SQLServerName;
string tableName = "Data." + tpt;
string connString = "server=" + server + ";uid=" + userName + ";pwd=" + password + ";database=" + database + ";Integrated Security=True;";
string databaseAndTable = "[" + database + "].[dbo].[" + tableName + "]"; /* [database].[dbo].[table], database = [Exchange.Issue.Data], table = [Data.TimePeriods(enum)] */
string sqlQueryString = "SELECT TOP " + recordCount.ToString() + " * FROM " + databaseAndTable + " WHERE [dtTime] >= '" + fromDate.ToString() + "' AND [dtTime] <= '" + toDate.ToString() + "' ORDER BY [dtTime] ASC";
using (SqlConnection sqlConnection = new SqlConnection(connString))
{
sqlConnection.Open();
using (SqlCommand sqlCmd = new SqlCommand(sqlQueryString, sqlConnection))
{
SqlDataReader rdr = sqlCmd.ExecuteReader();
while (rdr.Read())
{
ohlcArray.dtTime[idxArrayLoadPosition] = (DateTime)(rdr.GetSqlDateTime(0));
ohlcArray.High[idxArrayLoadPosition] = (double)(rdr.GetSqlDouble(1));
ohlcArray.Low[idxArrayLoadPosition] = (double)(rdr.GetSqlDouble(2));
ohlcArray.Open[idxArrayLoadPosition] = (double)(rdr.GetSqlDouble(3));
ohlcArray.Close[idxArrayLoadPosition] = (double)(rdr.GetSqlDouble(4));
ohlcArray.Volume[idxArrayLoadPosition] = int.Parse(rdr.GetSqlInt32(5).ToString());
returnDate = ohlcArray.dtTime[idxArrayLoadPosition];
idxArrayLoadPosition--;
}
rdr.Close();
}
sqlConnection.Close();
}
return returnDate;
}
Any ideas on which direction to go next to solve this?
Edit: more info ... the tables have this structure:
string cTable = "CREATE TABLE " + dbAndTable + " ([dtTime] [datetime] NOT NULL,[Bid] [float] NULL, [Ask] [float] NULL,[Volume] [float] NULL) ON [PRIMARY] ";
string cIndex = "CREATE INDEX " + indexName + " ON " + dbAndTable + " (dtTime)";<br/>
Should the [dTime] column be a primary key as opposed to just an index?
My friend your biggest problem is the implicit data conversion you are forcing sql server to do for you by converting from the strings "fromdate" and "todate" to the datetime datatype you established with your provided CREATE TABLE statement.
This is punishing processing running on such a large table. I believe you should
Modify the CREATE TABLE statement to include a primary key with a clustered index on an identity(1,1) column for your table.
Modify the CREATE INDEX using the INCLUDED option for the columns Bid,Ask and Volume.
Write a stored procedure passing as parameters your desired TOP, LOWER DATE FLOOR AND UPPER DATE CEILING AS datatypes INT, DATETIME, DATETIME.
Refactor your calling code to execute the stored procedure.
These are all well established best practices as I understand them.
Followup: I noticed that you are implementing paging after bringing all records back from the db ... instead implementing paging with SQL SERVER using FETCH NEXT will speed processing. Implement the necessary parameters on the stored procedure.
First thing we need to know the delay type:
Please run sp_whoisactive #get_plans=1. Google and download whoisactive if you didn't have one.
Then check the wait type column. It would tell you the wait types. Please post here we can take a look.
Without knowing the wait types, there are a few possible reasons:
1.
Usually after loading 25m rows, your buffer pool would be affected: existing cached data or compiled plans might be pushed out back to make room for loaded rows.
You might have async_network_io wait type, it does not necessarily mean you have network delay one of possible reason was that SQL SERVER is waiting for the client to process the data before sending more data.
What is the correct way to concatenate these elements?
Environment: C#-WinForm-Desktop.
In my app, I have a text file called config.ini. This file has values with this nomenclature:
csv_month_hour=value
The data are thus:
[csv]
csv_01_01=value // 24 hours a day in January
csv_01_02=value
csv_01_03=value
.
csv_02_01=value // 24 hours a day in February
csv_02_02=value
csv_02_03=value
.
// and so on
My app has this method called:
private void getMonthHour()
{
DateTime Xdt = DateTime.Now;
int Xmonth = Xdt.Month;
int Xhour = Xdt.Hour;
if (File.Exists("config.ini"))
{
IniFile ini = new IniFile("config.ini");
m3h = Convert.ToDouble(confg.csv["csv_" + Xmonth.ToString + "_" + Xhour.ToString]);
}
}
This method is called every seconds in timer_tick to check the month and time to use the corresponding value.
I need to know how to concatenate this:
m3h = Convert.ToDouble(confg.csv["csv_"+Xmonth.ToString+"_"+Xhour.ToString]);
An example of the code would be:
if (Xmonth==1&&Xhour==1){m3h = Convert.ToDouble(confg.csv.csv_01_01);}
else if (Xmonth==1&&Xhour==2){m3h = Convert.ToDouble(confg.csv.csv_01_02);}
else if (Xmonth==1&&Xhour==3){m3h = Convert.ToDouble(confg.csv.csv_01_03);}
// csv_month_hour in this case 01 is January.
else if (Xmonth==2&&Xhour==1){m3h = Convert.ToDouble(confg.csv.csv_02_01);}
// And so on, until December (12).
Putting aside the invalid syntax/compiler error from your method calls in your code, I suspect you're building the string but can't get the leading 0 in front of digits 0-9. So when Xmonth and Xhour are less than 10, your string is being built as "csv_1_1" instead of your intended "csv_01_01".
To instruct the ToString method to include a leading zero when needed, you can supply a custom numeric format string "00" (see the first entry in the link for "Zero placeholder").
Try using Xmonth.ToString("00") and Xhour.ToString("00"):
Monitor.malyt.m3h = Convert.ToDouble(confg.csv["csv_" +
Xmonth.ToString("00") + "_" +
Xhour.ToString("00")]);
This will output the values of Xmonth and Xhour with a single leading zero where needed.
I have a database table "Leave".
Its primary key is "LeaveNo".
The other fields in the database include "LeaveDate", "EmpID" , "EmpName" , etc
I want to use the dataset.tables["Leave"].Select() method to acquire only the leaves the employee has made in a single month.
The best way I figured was to use two intervals i.e mm/01/2012 and mm/31/2012
(I know some months have 30 days)
My current code is this and it does not work:
This was just a test code...
string moreThan = DateTime.Now.Month.ToString() + "01" + DateTime.Now.Year.ToString();
string lessThan = DateTime.Now.Month.ToString() + "31" + DateTime.Now.Year.ToString();
leavesthisMonth = LeaveDS.Tables["Leave"].Select("EmpID='" + txt_EmpID.Text + "' AND LeaveDate <= '" + "#" + lessThan + "#" + "'" + "AND LeaveDate >= '" + "#" + moreThan + "#" + "'");
Can someone help me with this ???
Working with DataSets and DataTables means that you are not bound (or anyway no more) to a database or specific SQL dialect.
The syntax used with the DataTable.Select() method is documented on MSDN.
For dates (obviously if the column type is REALLY DateTime) is used a syntax similar to MS Access (# delimited), so you can write:
var rows = dt.Select("LeaveDate > #2013-01-01#");
Is important to NOT use quotes (as in #Waqas code):
var rows = dt.Select("LeaveDate > '#2013-01-01#'"); // ERROR
because results in: System.FormatException: String was not recognized as a valid DateTime.
My advice is to use the yyyy-MM-dd format because is culture invariant by design.
A good way to build strings with a specific syntax is to use String.Format (or any variants) for clarity reasons, and let the DateTime type do all the specific date maths.
int month = 10;
var begin = new DateTime(DateTime.Today.Year,month,1); // first day of a month
var end = begin.AddMonths(1); // first day next month
var format = "LeaveDate >= #{0:yyyy-MM-dd}# AND LeaveDate < #{1:yyyy-MM-dd}#";
var condition = String.Format(format,begin,end);
rows = dt.Select(condition);
Notice your operator <= LeaveDate and >= LeaveDate
The operator it should be like >= LeaveDate and <= LeaveDate
The LeaveDate it should be like LeaveDate.ToString("dd MMM yyyy") or LeaveDate.ToString("dd/mm/yyyy")
leavesthisMonth = LeaveDS.Tables["Leave"].Select("EmpID='" + txt_EmpID.Text + "' AND (LeaveDate >= #" + lessThan + "# AND LeaveDate <= #" + moreThan + "#)");
You could make the whole thing a lot easier by using Date() in the SQL:
WHERE Month(LeaveDate)=Month(Date()) AND Year(LeaveDate)=Year(Date())
That removes all the date handling to MS Access and avoids a lot of problem.
the code from the last 700 is this
private string strsqlcommandBeta =
"select top 700 * from objectaer " +
" order by objectdate desc";
and the code for the first 10 words of the memo field called patronvalue17 should be this.
private string strsqlcommandBeta = "select LEFT(patronvalue17, INSTR(10, patronvalue17, " ") - 1) from objectaer " + " order by objectdate desc" + " ";
I don't know how to put this together
this code give the next error
select top 700 LEFT(patronvalue17, INSTR(10, patronvalue17, ' ') - 1) * from objectaer
Syntax error (missing operator) in query expression 'LEFT(patronvalue17, INSTR(10, patronvalue17, ' ') - 1) *'.
Scott found the syntactic bug. The reason why you're not getting the 10 words is because that's not how INSTR works. Its signature is
INSTR(Start_Posn, String_Being_Searched, Sought_String_Item, Compare_Type)
meaning you're looking up the first space after character 10 and then getting everything to the left of that. Normally that means you're getting 2-3 words out, rather than 10.
If you're getting all the memo text out that sounds very strange to me, but is indicative that InStr fails to match your space character and returns the index of the last character. I'd try supplying a comparetype.
http://www.techonthenet.com/access/functions/string/instr.php
You appear to be missing a ',' character between your two selected fields. -1), *