How to cache images from a SQl Server 2005 database call? - c#

I am having trouble using the output cache param in an aspnet 2.0 page directive. I am using a session variable to hold the value of the selected image. It seems that the dynamic control in a datalist, is not working when the output cache is set to true in the page directive. Is there a way to cache the images seperately to avoid using a page directive?
datalist code
" RepeatColumns="6" CellPadding="8" CellSpacing="8" GridLines="Both" SelectedItemStyle-BackColor="#33ff66" OnSelectedIndexChanged="dtlImages_SelectedIndexChanged" OnItemCommand="dtlImages_ItemCommand">
' Runat="server">
' ID="lblDescription" Font-Bold="True" Font-Size="12px" Font-Names="Arial">
code that retrieves the image from the database
protected void Page_Load(object sender, System.EventArgs e)
{
string strImageID = Request.QueryString["id"];
string wf4uConnect = System.Configuration.ConfigurationManager.ConnectionStrings["wf4uConnectionString"].ConnectionString;
System.Data.SqlClient.SqlConnection wf4uConn = new System.Data.SqlClient.SqlConnection(wf4uConnect);
System.Data.SqlClient.SqlCommand myCommand = new SqlCommand("Select ImageFile, ImageType from wf4u_ImageThumb Where ImageId =" + strImageID, wf4uConn);
wf4uConn.Open();
SqlDataReader byteReader = myCommand.ExecuteReader(CommandBehavior.CloseConnection);
while ((byteReader.Read()))
{
Response.BinaryWrite((byte [])byteReader.GetValue(0));
Response.ContentType = (string)byteReader.GetValue(1);
}
wf4uConn.Close();
}
I have implemented an http context object to cache the images as they're loaded into the webpage.
public ImageList (string clientName)
{
HttpContext context = HttpContext.Current;
if((context.Cache["ImageIdList" + clientName] == null))
{
string wf4uConnect = System.Configuration.ConfigurationManager.ConnectionStrings["wf4uConnectionString"].ConnectionString;
System.Data.SqlClient.SqlConnection wf4uConn = new System.Data.SqlClient.SqlConnection(wf4uConnect);
string queryStr = "SELECT ImageId FROM wf4u_imageThumb WHERE ClientName = #ClientName";
SqlCommand ImageIdComm = new System.Data.SqlClient.SqlCommand(queryStr, wf4uConn);
ImageIdComm.Parameters.Add("#ClientName", SqlDbType.VarChar).Value = clientName;
wf4uConn.Open();
SqlDataReader ImageIdReader = ImageIdComm.ExecuteReader();
if (ImageIdReader.Read())
{
_ImageId = ImageIdReader.GetInt32(0);
_ClientName = ImageIdReader.GetString(1);
context.Cache.Insert("ImageIdList" + clientName, this, null, DateTime.Now.AddSeconds(600), TimeSpan.Zero);
}
wf4uConn.Close();
}
else
{
ImageList list = (ImageList)context.Cache["ImageIdList" + clientName];
_ImageId = list.ImageId;
_ClientName = list.ClientName;
}
}
any suggestions would be welcomed.

You can use Cache object see: ASP.NET Caching: Techniques and Best Practices

You can implement an async handler like this:
using System;
using System.Web;
using System.Data.SqlClient;
public class FileHandler : IHttpAsyncHandler
{
#region Variables
private HttpContext _currentContext = null;
protected SqlCommand _command = null;
protected delegate void DoNothingDelegate();
#endregion
public void ProcessRequest(HttpContext context)
{
}
public void DoNothing()
{
}
public bool IsReusable
{
get { return false; }
}
#region IHttpAsyncHandler Members
public IAsyncResult BeginProcessRequest(HttpContext context, AsyncCallback cb, object extraData)
{
_currentContext = context;
int imageId;
if (int.TryParse(context.Request.QueryString["imageId"], out imageId))
{
// begin get file - make you own
// return FileController.BeginGetFile(out _command, cb, extraData, imageId);
}
else
{
DoNothingDelegate doNothingDelegate = new DoNothingDelegate(DoNothing);
return doNothingDelegate.BeginInvoke(cb, extraData);
}
}
public void EndProcessRequest(IAsyncResult result)
{
File file = null;
if (null != _command)
{
try
{
// end get file - make your own
// file = FileController.EndGetFile(_command, result);
}
catch { }
}
if (null != file)
{
// in my case File entity may be stored as a binary data
//or as an URL
// here is URL processing - redirect only
if (null == file.Data)
{
_currentContext.Response.Redirect(file.Path);
}
else
{
_currentContext.Response.ContentType = file.ContentType;
_currentContext.Response.BinaryWrite(file.Data);
_currentContext.Response.AddHeader("content-disposition", "filename=\"" + file.Name + (file.Name.Contains(file.Extension) ? string.Empty : file.Extension) + "\"" );
_currentContext.Response.AddHeader("content-length", (file.Data == null ? "0" : file.Data.Length.ToString()));
}
_currentContext.Response.Cache.SetCacheability(HttpCacheability.Private);
_currentContext.Response.Cache.SetExpires(DateTime.Now);
_currentContext.Response.Cache.SetSlidingExpiration(false);
_currentContext.Response.Cache.SetValidUntilExpires(false);
_currentContext.Response.Flush();
}
else
{
throw new HttpException(404, HttpContext.GetGlobalResourceObject("Image", "NotFound") as string);
}
}
#endregion
}
And set necessary values to the cache headers.
EDIT: Guess, in this case it would be better to use headers instead of using Cache object.

The way I do the images-from-the-database thing is to use a HTTP handler which reads the bits from the database, and sets the appropriate client-side caching headers in the response. I also support the If-Modified-Since header to suppress sending the blob if not required.
Then the page just has links to the handler with an image ID (must be careful to validate that the ID is an integer or whatever ID format you want to use - public sites get numerous SQL injection-style hack attempts.
For my current design the ID changes if the image is modified, so the caching time can be set to something long, e.g. a month or a year.

Related

how to load from access database to datagridview depends on combobox selection

i am getting this error
System.Data.EvaluateException: 'Cannot find column [Server].'
Depends on the combobox slection i want to load data from access database
[Server data from database that depends on that server field
i want output like this i want to load data in datagrid]2
Help me to load data from access database to datagrid depends on the combobox
namespace CatiaLicenseUsageReport
{
public partial class FrmCatiaLicenseUsageReport : Form
{
FileIO objFileIO ;
Utilities objUtilities ;
DBIO objDBIO ;
private string logfilepath = "";
private string DBPath = "";
public FrmCatiaLicenseUsageReport()
{
InitializeComponent();
Initialize();
}
}
private void cmbServer_SelectedIndexChanged(object sender, EventArgs e)
{
DataView dv = new DataView(objDBIO.ImportLicenseUsageDataFromDB(DBPath));
dv.RowFilter = string.Format("Server LIKE '%{0}%'", cmbServer.SelectedItem.ToString());
LicenseAllctnGridView.DataSource = dv;
}
//Database connection code
namespace CatiaLicenseUsageReport.IO
{
public class DBIO
{
/// <summary>
///
/// </summary>
/// <returns></returns>
#region Read DataBase File Function
///
public DBIO()
{
_cnStr = string.Empty;
_cmd = null;
_cnn = null;
}
public void Reset()
{
_cnStr = string.Empty;
_cmd = null;
_cnn = null;
}
private string _cnStr;
private OleDbCommand _cmd;
private OleDbConnection _cnn;
private OleDbDataAdapter _oda;
private OleDbCommandBuilder _cmdbl;
DataTable resultDt = new DataTable();
/// <summary>
/// Get Coonection for given file
/// </summary>
/// <param name="fileName"></param>
public bool GetConnectionToDb(string dbName)
{
try
{
_cmd = null;
if (_cnn != null)
_cnn.Close();
_cnStr = GetConnectionString(dbName);
_cnn = new OleDbConnection(_cnStr);
_cnn.Open();
return true;
}
catch (OutOfMemoryException)
{
return false;
}
catch (Exception)
{
return false;
}
}
private string GetConnectionString(string dbName)
{
string cnnStr;
string provider;
provider = "Microsoft.Jet.OLEDB.4.0";
cnnStr = "Provider=" + provider + ";Data Source=" + dbName + ";";
return cnnStr;
}
public DataTable ImportLicenseUsageDataFromDB(string dbName)
{
string query;
try
{
GetConnectionToDb(dbName);
//Retrive Data
_oda = new OleDbDataAdapter();
query = String.Format("select Department,AllottedLicense from AllottedLicense where Server = 'LUM' OR Server = 'DSLS'");
_cmd = new OleDbCommand(query, _cnn);
_oda.SelectCommand = _cmd;
_oda.Fill(resultDt);
}
catch (Exception ex)
{
Console.WriteLine("Error in function ImportLicenseUsageDataFromDB" + ex.StackTrace);
Logger.OutLine("Error in function ImportLicenseUsageDataFromDB" + ex.StackTrace);
throw ex;
}
finally
{
_cmd = null;
if (_cnn != null)
_cnn.Close();
//cnn = null;
}
return resultDt;
}
Make your query
select Server,Department,AllottedLicense from AllottedLicense where Server = 'LUM' OR Server = 'DSLS'
Or if you want to choose more than those two servers
select server,Department,AllottedLicense from AllottedLicense
RowFilter is a client side thing, so you have to have the Server column in the data table data in order for the DataView to be able to find it and filter on it. Setting a RowFilter does NOT alter any query done on the server, it just filters data already downloaded from the server, which is why that data needs to contain the column you want to filter on
To be honest, if you’re going to keep that code, you might as well throw the RowFilter idea away and just use a parameterized SQL query, given that you re download from the server every time you make a new DataView- the idea of RowFilter is that you download X records from the server once and then change the RowFilter to change what the client shows. More like this, pseudocode:
class WhateverForm: Form{
public WhateverForm(){
someDataGrid.DataSource = db.DownloadAllDataAsDatatable();
}
private void SomeCombo_SelectedIndexChanged(object sender, EventArgs e){
(someDataGrid.DataSource as DataTable).DefaultView.RowFilter = $”[Server] = ‘{someCombo.SelectedItem}’”;
}
}
Stash the data table you get from your query method into the grid view data source, get it out when you need and cast it to a data table , then set the RowFilter of the DefaultView (which is a DataView) of the table. When a grid is bound to a data table actually binds to the DefauktView so setting the RowFilter on it changes the grid contents. You don’t need to use LIKE if your combo contains exact server names, which I guess it should - using like will cause eg all the Ukraine servers to show up (as well as the UK ones) if you choose UK..

I Want to make function in different class(outside my form) for my first-Next-Previous-Last buttons for my All forms with same buttons

I Want to make function in different class(outside my form) for my first-Next-Previous-Last buttons for my All forms with same buttons .
How should I do it ? no Idea to access textboxes in other class.
My Code is here for Next button !!
<pre lang="c#">
private void CmdNext_Click(object sender, EventArgs e) {
ConStr Constring = new ConStr();
String CS = Constring.ConString("DBCS");
using (SqlConnection con = new SqlConnection(CS))
{
SqlDataAdapter DA = new SqlDataAdapter("Select * from FeesHead order by ID", con);
DataSet DS = new DataSet();
DA.Fill(DS, "FeesHead");
if ( i < DS.Tables["FeesHead"].Rows.Count-1)
{
i++;
this.TxtID.Text = DS.Tables["FeesHead"].Rows[i]["ID"].ToString();
this.TxtFees.Text = DS.Tables["FeesHead"].Rows[i]["Fees"].ToString();
this.TxtFeesHead.Text = DS.Tables["FeesHead"].Rows[i]["FeesHead"].ToString();
this.TxtRemarks.Text = DS.Tables["FeesHead"].Rows[i]["Remarks"].ToString();
if (i == DS.Tables["FeesHead"].Rows.Count - 1)
{
MessageBox.Show("This is Last Record");
}<pre lang="c#">
I am Trying Class library for above procedure My code is here !! I need some corrections !! I feed difficulty to access my form text box here so I left blank this space !!
//
`public void move_prev(String Con_String, String Table)
{
{
using (SqlConnection con = new SqlConnection(Con_String))
{
SqlDataAdapter DA = new SqlDataAdapter("Select * from " + Table +"", con);
DataSet DS = new DataSet();
DA.Fill(DS, Table);
for (int j=0; j<= DS.Tables[Table].Columns.Count;j++ )
{
int i = 0;
String name_of_column = DS.Tables[Table].Columns[j].ColumnName;
if (i == DS.Tables[Table].Rows.Count - 1 || i != 0)
{
i--;
= DS.Tables[Table].Rows[i][name_of_column].ToString();
if (i == 0)
{
MessageBox.Show("This is First Record");
}
}
}
}}
}
}
}
`//
You should try to have business logic in a separate class from the UI logic. I'm sure you've encountered the terms Model View Controller (MVC), Model View Presenter (MVP) and Model View, ViewModel (MVVM), all of which are approaches to achieving this.
In the context of this question, a simplified version of this can be achieved by encapsulating the 'first-prev-next-last' last logic in a business logic class and attaching an instance of it to the form class, (where attachment might be done via a factory class or method.) The business logic class exposes methods and properties that the UI form class can access.
public interface INavigation
{
// Property mapping field names to values
public Dictionary<string,string> CurrentRecordFields;
public void FirstRecord(void);
// ... plus all the other navigation methods.
}
The use of an interface here has several advantages: you can create a mock class for unit testing purposes, or you might have different implementations of the interface (for different tables in the database for example) any of which can re-use the same UI.
public class Navigation : INavigation
{
// Properties from INavigator
public Dictionary<string,string> CurrentRecordFields {get; set;}
// private fields
private string conString;
private string tableName;
// Constructor requires that the connection string and
// required table name be passed as arguments.
public Navigation(String Con_String, String Table)
{
conString = Con_String;
tableName = Table;
CurrentRecordFields = new Dictionary<string,string>();
}
// Note - this should really do some kind of error checking
// and exception handling.
public void FirstRecord(void)
{
using (SqlConnection con = new SqlConnection(conString))
{
SqlDataAdapter DA = new SqlDataAdapter("Select * from " + tableName +"", con);
DataSet DS = new DataSet();
DA.Fill(DS, tableName);
// Populate the Dictionary for the first row.
CurrentRecordFields.Empty();
foreach(var column in DS.Tables[tableName].Columns)
{
string columnName = column.ColumnName;
CurrentRecordFields.Add(columnName,
DS.Tables[tableName].Rows[0][columnName].ToString());
}
}
}
// ... and the rest of the navigation methods: prev, next, last, etc.
}
public partial class NavigationForm
{
private INavigation Navigator {get; set;}
public NavigationForm(INavigation navigator) :
base()
{
this.Navigator = navigator;
}
private void CmdFirst_Click(object sender, EventArgs e)
{
// Use the business logic class to do the DB work.
// See note above regarding error checking and exception handling.
Navigator.FirstRecord();
// Update the UI from the business logic properties.
this.TxtID.Text = Navigator.CurrentRecordFields["ID"];
this.TxtFees.Text = Navigator.CurrentRecordFields["Fees"];
this.TxtFeesHead.Text = Navigator.CurrentRecordFields["FeesHead"];
this.TxtRemarks.Text = Navigator.CurrentRecordFields["Remarks"];
}
// Etc., for the other form methods.
}
Then, when you create the form, you also create an instance of the Navigation business logic class and pass it to the form class constructor.
// In the main form or application
private void OnNavigateThroughRecords()
{
// Create the navigator for the required table.
INavigation navigator = new Navigation(this.ConectionString, this.SelectedTableName);
Form navigationForm = new NavigationForm(navigator);
}
This isn't a complete example, but it should be enough to get you started. For more information and plentiful examples, read the many rich resources on the various three-tiered architectures (MVC, MVP and MVVM being amongst the most well known).
EDIT
Changed the properties of the Navigator so that an unspecified collection of fields can be presented by the use of the Dictionary class. This is populated by the Navigator class methods with each entry containing the name of the column as the Key and the value for the record as the Value.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.SqlClient;
using System.Configuration;
using System.Windows.Forms;
using System.Data;
namespace APMC
{
class Action_Btn
{
public void Navigation_Btns(String Con_Str, String Tab_Name, Control form, String Btn_Name,int ID)
{
int i = ID;
List<Control> TB_List = new List<Control>();
foreach (Control childcontrol in form.Controls)
{
if (childcontrol is GroupBox)
{
foreach (Control cc in childcontrol.Controls)
{
if (cc is TextBox)
{
TB_List.Add(cc);
}
}
}
}
using (SqlConnection con = new SqlConnection(Con_Str))
{
SqlDataAdapter da = new SqlDataAdapter("Select * from " + Tab_Name + "", con);
DataSet ds = new DataSet();
da.Fill(ds, Tab_Name);
if (Btn_Name == "CmdFirst")
{
if (ds.Tables[Tab_Name].Rows.Count > 0)
{
i = 0;
foreach (Control Txt_Name in TB_List)
{
String field_name = Txt_Name.Name.Substring(3);
Txt_Name.Text = ds.Tables[Tab_Name].Rows[i][field_name].ToString();
}
}
if (i == 0 )
{
MessageBox.Show("This is First Record");
}
}
if (Btn_Name == "CmdNext")
{
if (i < ds.Tables[Tab_Name].Rows.Count )
{
foreach (Control Txt_Name in TB_List)
{
String field_name = Txt_Name.Name.Substring(3);
Txt_Name.Text = ds.Tables[Tab_Name].Rows[i][field_name].ToString();
}
}
if (i == ds.Tables[Tab_Name].Rows.Count)
{
MessageBox.Show("This is Last Record");
}
}
if (Btn_Name == "CmdPrev")
{
if (i == ds.Tables[Tab_Name].Rows.Count || i != 1)
{
i = i - 2;
foreach (Control Txt_Name in TB_List)
{
String field_name = Txt_Name.Name.Substring(3);
Txt_Name.Text = ds.Tables[Tab_Name].Rows[i][field_name].ToString();
}
}
if (i == 0)
{
MessageBox.Show("This is First Record");
}
}
if (Btn_Name == "CmdLast")
{
if (ds.Tables[Tab_Name].Rows.Count - 1 >= i)
{
i = ds.Tables[Tab_Name].Rows.Count - 1;
foreach (Control Txt_Name in TB_List)
{
String field_name = Txt_Name.Name.Substring(3);
Txt_Name.Text = ds.Tables[Tab_Name].Rows[i][field_name].ToString();
}
}
if (i == ds.Tables[Tab_Name].Rows.Count)
{
MessageBox.Show("This is Last Record");
}
}
}
}
}
}

How to take a CSV field and write to columns in SQL

I have the following code which takes a CSV and writes to a console:
using (CsvReader csv = new CsvReader(
new StreamReader("data.csv"), true))
{
// missing fields will not throw an exception,
// but will instead be treated as if there was a null value
csv.MissingFieldAction = MissingFieldAction.ReplaceByNull;
// to replace by "" instead, then use the following action:
//csv.MissingFieldAction = MissingFieldAction.ReplaceByEmpty;
int fieldCount = csv.FieldCount;
string[] headers = csv.GetFieldHeaders();
while (csv.ReadNextRecord())
{
for (int i = 0; i < fieldCount; i++)
Console.Write(string.Format("{0} = {1};",
headers[i],
csv[i] == null ? "MISSING" : csv[i]));
Console.WriteLine();
}
}
The CSV file has 7 headers for which I have 7 columns in my SQL table.
What is the best way to take each csv[i] and write to a row for each column and then move to the next row?
I tried to add the ccsv[i] to a string array but that didn't work.
I also tried the following:
SqlCommand sql = new SqlCommand("INSERT INTO table1 [" + csv[i] + "]", mysqlconnectionstring);
sql.ExecuteNonQuery();
My table (table1) is like this:
name address city zipcode phone fax device
your problem is simple but I will take it one step further and let you know a better way to approach the issue.
when you have a problem to sold, always break it down into parts and apply each part in each own method. For example, in your case:
1 - read from the file
2 - create a sql query
3 - run the query
and you can even add validation to the file (imagine your file does not even have 7 fields in one or more lines...) and the example below it to be taken, only if your file never passes around 500 lines, as if it does normally you should consider to use a SQL statement that takes your file directly in to the database, it's called bulk insert
1 - read from file:
I would use a List<string> to hold the line entries and I always use StreamReader to read from text files.
using (StreamReader sr = File.OpenText(this.CsvPath))
{
while ((line = sr.ReadLine()) != null)
{
splittedLine = line.Split(new string[] { this.Separator }, StringSplitOptions.None);
if (iLine == 0 && this.HasHeader)
// header line
this.Header = splittedLine;
else
this.Lines.Add(splittedLine);
iLine++;
}
}
2 - generate the sql
foreach (var line in this.Lines)
{
string entries = string.Concat("'", string.Join("','", line))
.TrimEnd('\'').TrimEnd(','); // remove last ",'"
this.Query.Add(string.Format(this.LineTemplate, entries));
}
3 - run the query
SqlCommand sql = new SqlCommand(string.Join("", query), mysqlconnectionstring);
sql.ExecuteNonQuery();
having some fun I end up doing the solution and you can download it here, the output is:
The code can be found here. It needs more tweaks but I will left that for others. Solution written in C#, VS 2013.
The ExtractCsvIntoSql class is as follows:
public class ExtractCsvIntoSql
{
private string CsvPath, Separator;
private bool HasHeader;
private List<string[]> Lines;
private List<string> Query;
/// <summary>
/// Header content of the CSV File
/// </summary>
public string[] Header { get; private set; }
/// <summary>
/// Template to be used in each INSERT Query statement
/// </summary>
public string LineTemplate { get; set; }
public ExtractCsvIntoSql(string csvPath, string separator, bool hasHeader = false)
{
this.CsvPath = csvPath;
this.Separator = separator;
this.HasHeader = hasHeader;
this.Lines = new List<string[]>();
// you can also set this
this.LineTemplate = "INSERT INTO [table1] SELECT ({0});";
}
/// <summary>
/// Generates the SQL Query
/// </summary>
/// <returns></returns>
public List<string> Generate()
{
if(this.CsvPath == null)
throw new ArgumentException("CSV Path can't be empty");
// extract csv into object
Extract();
// generate sql query
GenerateQuery();
return this.Query;
}
private void Extract()
{
string line;
string[] splittedLine;
int iLine = 0;
try
{
using (StreamReader sr = File.OpenText(this.CsvPath))
{
while ((line = sr.ReadLine()) != null)
{
splittedLine = line.Split(new string[] { this.Separator }, StringSplitOptions.None);
if (iLine == 0 && this.HasHeader)
// header line
this.Header = splittedLine;
else
this.Lines.Add(splittedLine);
iLine++;
}
}
}
catch (Exception ex)
{
if(ex.InnerException != null)
while (ex.InnerException != null)
ex = ex.InnerException;
throw ex;
}
// Lines will have all rows and each row, the column entry
}
private void GenerateQuery()
{
foreach (var line in this.Lines)
{
string entries = string.Concat("'", string.Join("','", line))
.TrimEnd('\'').TrimEnd(','); // remove last ",'"
this.Query.Add(string.Format(this.LineTemplate, entries));
}
}
}
and you can run it as:
class Program
{
static void Main(string[] args)
{
string file = Ask("What is the CSV file path? (full path)");
string separator = Ask("What is the current separator? (; or ,)");
var extract = new ExtractCsvIntoSql(file, separator);
var sql = extract.Generate();
Output(sql);
}
private static void Output(IEnumerable<string> sql)
{
foreach(var query in sql)
Console.WriteLine(query);
Console.WriteLine("*******************************************");
Console.Write("END ");
Console.ReadLine();
}
private static string Ask(string question)
{
Console.WriteLine("*******************************************");
Console.WriteLine(question);
Console.Write("= ");
return Console.ReadLine();
}
}
Usually i like to be a bit more generic so i'll try to explain a very basic flow i use from time to time:
I don't like the hard coded attitude so even if your code will work it will be dedicated specifically to one type. I prefer i simple reflection, first to understand what DTO is it and then to understand what repository should i use to manipulate it:
For example:
public class ImportProvider
{
private readonly string _path;
private readonly ObjectResolver _objectResolver;
public ImportProvider(string path)
{
_path = path;
_objectResolver = new ObjectResolver();
}
public void Import()
{
var filePaths = Directory.GetFiles(_path, "*.csv");
foreach (var filePath in filePaths)
{
var fileName = Path.GetFileName(filePath);
var className = fileName.Remove(fileName.Length-4);
using (var reader = new CsvFileReader(filePath))
{
var row = new CsvRow();
var repository = (DaoBase)_objectResolver.Resolve("DAL.Repository", className + "Dao");
while (reader.ReadRow(row))
{
var dtoInstance = (DtoBase)_objectResolver.Resolve("DAL.DTO", className + "Dto");
dtoInstance.FillInstance(row.ToArray());
repository.Save(dtoInstance);
}
}
}
}
}
Above is a very basic class responsible importing the data. Nevertheless of how this piece of code parsing CSV files (CsvFileReader), the important part is thata "CsvRow" is a simple List.
Below is the implementation of the ObjectResolver:
public class ObjectResolver
{
private readonly Assembly _myDal;
public ObjectResolver()
{
_myDal = Assembly.Load("DAL");
}
public object Resolve(string nameSpace, string name)
{
var myLoadClass = _myDal.GetType(nameSpace + "." + name);
return Activator.CreateInstance(myLoadClass);
}
}
The idea is to simple follow a naming convetion, in my case is using a "Dto" suffix for reflecting the instances, and "Dao" suffix for reflecting the responsible dao. The full name of the Dto or the Dao can be taken from the csv name or from the header (as you wish)
Next step is filling the Dto, each dto or implements the following simple abstract:
public abstract class DtoBase
{
public abstract void FillInstance(params string[] parameters);
}
Since each Dto "knows" his structure (just like you knew to create an appropriate table in the database), it can easily implement the FillInstanceMethod, here is a simple Dto example:
public class ProductDto : DtoBase
{
public int ProductId { get; set; }
public double Weight { get; set; }
public int FamilyId { get; set; }
public override void FillInstance(params string[] parameters)
{
ProductId = int.Parse(parameters[0]);
Weight = double.Parse(parameters[1]);
FamilyId = int.Parse(parameters[2]);
}
}
After you have your Dto filled with data you should find the appropriate Dao to handle it
which is basically happens in reflection in this line of the Import() method:
var repository = (DaoBase)_objectResolver.Resolve("DAL.Repository", className + "Dao");
In my case the Dao implements an abstract base class - but it's not that relevant to your problem, your DaoBase can be a simple abstract with a single Save() method.
This way you have a dedicated Dao to CRUD your Dto's - each Dao simply knows how to save for its relevant Dto. Below is the corresponding ProductDao to the ProductDto:
public class ProductDao : DaoBase
{
private const string InsertProductQuery = #"SET foreign_key_checks = 0;
Insert into product (productID, weight, familyID)
VALUES (#productId, #weight, #familyId);
SET foreign_key_checks = 1;";
public override void Save(DtoBase dto)
{
var productToSave = dto as ProductDto;
var saveproductCommand = GetDbCommand(InsertProductQuery);
if (productToSave != null)
{
saveproductCommand.Parameters.Add(CreateParameter("#productId", productToSave.ProductId));
saveproductCommand.Parameters.Add(CreateParameter("#weight", productToSave.Weight));
saveproductCommand.Parameters.Add(CreateParameter("#familyId", productToSave.FamilyId));
ExecuteNonQuery(ref saveproductCommand);
}
}
}
Please ignore the CreateParameter() method, since it's an abstraction from the base classs. you can just use a CreateSqlParameter or CreateDataParameter etc.
Just notice, it's a real naive implementation - you can easily remodel it better, depends on your needs.
From the first impression of your questionc I guess you would be having hugely number of records (more than lacs). If yes I would consider the SQL bulk copies an option. If the record would be less go ahead single record. Insert. The reason for you insert not working is u not providing all the columns of the table and also there's some syntax error.

Duplicate entries on server response .net

Scenario
One windows service polls a url every two minutes to retrieve certain data.
If any data has been added since the previous call, the data is retrieved and stored otherwise the loop carries on.
Issue
Sometimes a request takes more than two minutes to return a response.
When this happens, the next request is still made and finds new data, since the previous request hasn't return a response yet
This results in duplicate entries when the data is stored.
What I've tried
I tried to handle that by using a boolean like so:
Boolean InProgress = true;
foreach (var item in Lists)
{
\\Make a request and return new data (if any)
InProgress = false;
if (InProgress = false)
{
\\Store new data
}
}
This doesn't solve the issue. I believe I'm using the boolean in wrong place, but I'm not sure where it should.
This is the loop that makes the request and store the data
void serviceTimer_Elapsed(object sender, ElapsedEventArgs e)
{
try
{
Data getCredentials = new Data();
DataTable credentials = getCredentials.loadCredentials();
Boolean InProgress = true;
for (int i = 0; i < credentials.Rows.Count; i++)
{
if (credentials != null)
{
var PBranchID = (int)credentials.Rows[i]["PortalBranchID"];
var negRef = (int)credentials.Rows[i]["NegotiatorRef"];
var Username = credentials.Rows[i]["Username"].ToString();
var Password = credentials.Rows[i]["Password"].ToString();
var Domain = credentials.Rows[i]["Domain"].ToString();
var FooCompanyBaseUrl = "https://" + Domain + ".FooCompany.com/";
Data getCalls = new Data();
DataTable calls = getCalls.loadCalls(PBranchID);
//If it's not the first call
if (calls != null && calls.Rows.Count > 0)
{
//Makes a call
DateTime CreatedSince = DateTime.SpecifyKind((DateTime)calls.Rows[0]["LastSuccessOn"], DateTimeKind.Local);
string IssueListUrl = FooCompany.WebApi.V2.URLs.Issues(BaseUrl, null, CreatedSince.ToUniversalTime(), null);
FooCompany.WebApi.V2.DTO.PrevNextPagedList resultIssueList;
resultIssueList = FooCompany.WebApi.Client.Helper.Utils.Getter<Foocompany.WebApi.V2.DTO.PrevNextPagedList>(IssueListUrl, Username, Password);
InProgress = false;
if (InProgress == false)
{
if (resultIssueList.Items.Count > 0)
{
//If call returns new issues, save call
Data saveCalls = new Data();
saveCalls.saveCalls(PBranchID);
foreach (var item in resultIssueList.Items)
{
var Issue = FooCompany.WebApi.Client.Helper.Utils.Getter<FooCompany.WebApi.V2.DTO.Issue>(item, Username, Password);
string TenantSurname = Issue.Surname;
string TenantEmail = Issue.EmailAddress;
Data tenants = new Data();
int tenantPropRef = Convert.ToInt32(tenants.loadTenantPropRef(PBranchID, TenantSurname, TenantEmail));
Data Properties = new Data();
DataTable propAddress = Properties.loadPropAddress(PBranchID, tenantPropRef);
var Address1 = propAddress.Rows[0]["Address1"];
var Address2 = propAddress.Rows[0]["Address2"];
var AddressFolder = Address1 + "," + Address2;
if (!Directory.Exists("path"))
{
Directory.CreateDirectory("path");
}
string ReportPDFDestination = "path";
if (File.Exists(ReportPDFDestination))
{
File.Delete(ReportPDFDestination);
}
FooCompany.WebApi.Client.Helper.Utils.DownloadFileAuthenticated(FooCompany.WebApi.V2.URLs.IssueReport(BaseUrl, Issue.Id), Username, Password, ReportPDFDestination);
//Store data
}
IssueListUrl = resultIssueList.NextURL;
}
}
}
else
{
continue;
}
}
}
catch (Exception ex)
{
//write to log
}
}
Question
I'm sure there is a better way than a boolean.
Could anyone advice a different method to handle the issue properly?
Thanks.
Solution
I ended up using a combination of both Thomas and Mason suggestions. I wrapped a lock statement around the main function of my windows service and used a boolean inside the function section that makes the call to the remote server.
Tested many times and it's error free.
You seems to have a problem of synchronisation, just surround the code that iterate though the List with a lock, and you will be fine.
public class MyClass{
private readonly object internalLock= new object();
private bool AlreadyRunning { get; set; }
void serviceTimer_Elapsed(object sender, ElapsedEventArgs e)
{
if(AlreadyRunning){
return;
}
try{
lock(internalLock){
Thread.MemoryBarrier();
if(AlreadyRunning){
return;
}
AlreadyRunning = true;
...Do all the things...
}
}
catch(Exception e){
..Exception handling
}
finally
{
AlreadyRunning = false;
}
}
bool InProgress=false;
void serviceTimer_Elapsed(object sender, ElapsedEventArgs e)
{
if(!InProgress)
{
InProgress=true;
//retrieve data
InProgress=false;
}
}
Your InProgress variable needs to be declared outside the event handler. When you enter the method, check to see if it's already running. If it is, then we do nothing. If it's not running, then we say it's running, retrieve our data, then reset our flag to say we've finished running.
You'll probably need to add appropriate locks for thread safety, similar to Thomas's answer.

Using an ObjectDataSource with a GridView in a dynamic scenario

I have a search page that is tasked with searching 3.5 million records for individuals based on their name, customer ID, address, etc. The queries range from complex to simple.
Currently, this code relies on a SqlDataSource and a GridView. When a user types a serach term in and presses enter, the TextBoxChanged even runs a Search(term, type) function that changes the query that the SqlDataSource uses, adds the parameters, and rebinds the GridView.
It works well, but I've become obsessed with rewriting the code more efficiently. I want the paging to be done by SQL Server instead of the inefficiencies of a SqlDataSource in DataSet mode.
Enter the ObjectDataSource. Caveat: I have never used one before today.
I have spent the better part of the day putting together this class:
using System;
using System.Data;
using System.Data.SqlClient;
using System.Configuration;
using System.Text;
using System.Web;
using System.Web.Security;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using System.Web.UI.HtmlControls;
/// <summary>
/// Summary description for MultiSearchData
/// </summary>
public class MultiSearchData
{
private string _connectionString = string.Empty;
private string _sortColumns = string.Empty;
private string _selectQuery = string.Empty;
private int _lastUpdate;
private int _lastRowCountUpdate;
private int _lastRowCount;
private SqlParameterCollection _sqlParams;
public MultiSearchData()
{
}
private void UpdateDate()
{
_lastUpdate = (int)(DateTime.UtcNow - new DateTime(1970, 1, 1)).TotalSeconds;
}
private string ReplaceFirst(string text, string search, string replace)
{
int pos = text.IndexOf(search);
if (pos < 0)
{
return text;
}
return text.Substring(0, pos) + replace + text.Substring(pos + search.Length);
}
public string SortColumns
{
get { return _sortColumns; }
set { _sortColumns = value; }
}
public SqlParameterCollection SqlParams
{
get { return _sqlParams; }
set { _sqlParams = value; }
}
public string ConnectionString
{
get { return _connectionString; }
set { _connectionString = value; }
}
public string SelectQuery
{
get { return _selectQuery; }
set
{
if (value != _selectQuery)
{
_selectQuery = value;
UpdateDate();
}
}
}
public DataTable GetFullDataTable()
{
return GetDataTable(AssembleSelectSql());
}
public DataTable GetPagedDataTable(int startRow, int pageSize, string sortColumns)
{
if (sortColumns.Length > 0)
_sortColumns = sortColumns;
return GetDataTable(AssemblePagedSelectSql(startRow, pageSize));
}
public int GetRowCount()
{
if (_lastRowCountUpdate == _lastUpdate)
{
return _lastRowCount;
}
else
{
string strCountQuery = _selectQuery.Remove(7, _selectQuery.IndexOf("FROM") - 7);
strCountQuery = strCountQuery.Replace("SELECT FROM", "SELECT COUNT(*) FROM");
using (SqlConnection conn = new SqlConnection(_connectionString))
{
conn.Open();
using (SqlCommand cmd = new SqlCommand(strCountQuery, conn))
{
if (_sqlParams.Count > 0)
{
foreach (SqlParameter param in _sqlParams)
{
cmd.Parameters.Add(param);
}
}
_lastRowCountUpdate = _lastUpdate;
_lastRowCount = (int)cmd.ExecuteScalar();
return _lastRowCount;
}
}
}
}
public DataTable GetDataTable(string sql)
{
DataTable dt = new DataTable();
using (SqlConnection conn = new SqlConnection(_connectionString))
{
using (SqlCommand GetCommand = new SqlCommand(sql, conn))
{
conn.Open();
if (_sqlParams.Count > 0)
{
foreach (SqlParameter param in _sqlParams)
{
GetCommand.Parameters.Add(param);
}
}
using (SqlDataReader dr = GetCommand.ExecuteReader())
{
dt.Load(dr);
conn.Close();
return dt;
}
}
}
}
private string AssembleSelectSql()
{
StringBuilder sql = new StringBuilder();
sql.Append(_selectQuery);
return sql.ToString();
}
private string AssemblePagedSelectSql(int startRow, int pageSize)
{
StringBuilder sql = new StringBuilder();
string originalQuery = ReplaceFirst(_selectQuery, "FROM", ", ROW_NUMBER() OVER (ORDER BY " + _sortColumns + ") AS ResultSetRowNumber FROM");
sql.Append("SELECT * FROM (");
sql.Append(originalQuery);
sql.Append(") AS PagedResults");
sql.AppendFormat(" WHERE ResultSetRowNumber > {0} AND ResultSetRowNumber <= {1}", startRow.ToString(), (startRow + pageSize).ToString());
return sql.ToString();
}
}
I don't know if it's pretty. It works. I give it a query in the ObjectCreating method:
protected void dataMultiSearchData_ObjectCreating(object sender, ObjectDataSourceEventArgs e)
{
MultiSearchData info;
info = Cache["MultiSearchDataObject"] as MultiSearchData;
if (null == info)
{
info = new MultiSearchData();
}
info.SortColumns = "filteredcontact.fullname";
info.ConnectionString = "Data Source=SERVER;Initial Catalog=TheDatabase;Integrated Security=sspi;Connection Timeout=60";
info.SelectQuery = #"SELECT filteredcontact.contactid,
filteredcontact.new_libertyid,
filteredcontact.fullname,
'' AS line1,
filteredcontact.emailaddress1,
filteredcontact.telephone1,
filteredcontact.birthdateutc AS birthdate,
filteredcontact.gendercodename
FROM filteredcontact
WHERE fullname LIKE 'Griffin%' AND filteredcontact.statecode = 0";
e.ObjectInstance = info;
}
protected void dataMultiSearchData_ObjectDisposing(object sender, ObjectDataSourceDisposingEventArgs e)
{
MultiSearchData info = e.ObjectInstance as MultiSearchData;
MultiSearchData temp = Cache["MultiSearchDataObject"] as MultiSearchData;
if (null == temp)
{
Cache.Insert("MultiSearchDataObject", info);
}
e.Cancel = true;
}
Once the class has the query, it wraps it in paging friendly SQL and we're off to the races. I've implemented caching so that it can skip some expensive queries. Etc.
My problem is, this completely breaks my pretty little Search(term, type) world. Having ot set the query in the ObjectCreating method is completely harshing my vibe.
I've been trying to think of a better way to do this all day, but I keep ending up with a really messy...do it all in ObjectCreating model that just turns my stomach.
How would you do this? How can I keep the efficiency of this new method whilst have the organizational simplicity of my former model?
Am I being too OCD?
I determined that it can't be done. Furthermore, after benchmarking this class I found it performed no better than a SqlDataSource but was much more difficult to maintain.
Thus I abandoned this project. I hope someone finds this code useful at some point though.

Categories

Resources